← all repositories
AeternaLabsHQ/pullmd

A translator between the messy web and your token-starved agent

PullMD is a self-hosted service that turns URLs and files into clean, token-lean Markdown — because raw HTML is a context-window crime.

★507 stars JavaScript Data ToolingCoding Assistants
pullmd
Collecting fresh signals — velocity needs a few days of history.
collecting data…
star history

What it does

Give it a URL or upload a file and it hands back clean Markdown: web pages, PDFs, Office docs, EPUB, images, audio, and YouTube transcripts. It runs as a self-hosted service with a web UI, REST API, MCP server, and a Claude Code skill. Metadata lives in YAML frontmatter so the body stays pure content.

The interesting bit

The extraction is a fallback chain — Cloudflare’s native Markdown, then Readability and Trafilatura with the best result winning, then headless Chromium for JavaScript-heavy sites. There’s also query-scoped extraction: pass a question and get only the relevant sections. The README’s own numbers show a Reddit thread that plain curl can’t even fetch (bot check) coming back as ~6,400 tokens, or ~640 with a query attached.

Key highlights

  • Reddit and Hacker News threads with full comment trees, depth and count configurable
  • YouTube transcripts with clickable timecodes, no API key needed
  • Live share links (/s/:id) that re-fetch stale caches and serve the last good snapshot if the source dies
  • Every sidecar is optional — drop Playwright or MarkItDown and it degrades gracefully
  • SSRF protection, rate limits, and auth modes from open homelab to multi-user

Caveats

  • The Playwright sidecar adds ~3.7 GB of image cache (Chromium, Firefox, WebKit) — optional, but the JS-heavy-site fallback is why you’d want it
  • Reverse-proxy setups need PULLMD_TRUST_PROXY configured correctly or all clients share one rate-limit bucket
  • Token-savings numbers are the project’s own measurements on three pages, not an independent benchmark

Verdict

If you run agents that read the web — especially via Claude Code or MCP — this is a genuinely useful middleman. If you just need one-off HTML-to-Markdown, a library will do the job with less ceremony.

Frequently asked

What is AeternaLabsHQ/pullmd?
PullMD is a self-hosted service that turns URLs and files into clean, token-lean Markdown — because raw HTML is a context-window crime.
Is pullmd open source?
Yes — AeternaLabsHQ/pullmd is open source, released under the AGPL-3.0 license.
What language is pullmd written in?
AeternaLabsHQ/pullmd is primarily written in JavaScript.
How popular is pullmd?
AeternaLabsHQ/pullmd has 507 stars on GitHub.
Where can I find pullmd?
AeternaLabsHQ/pullmd is on GitHub at https://github.com/AeternaLabsHQ/pullmd.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.