Scraping infrastructure that admits the job is mostly plumbing

What it does

Reader is a TypeScript scraping and crawling engine built on top of Ulixee Hero. It exposes three APIs: scrape() for URL-to-markdown conversion, crawl() for BFS site discovery, and browser() for stealthed Chrome sessions you drive via Playwright or Puppeteer. The self-hosted version runs a browser pool with auto-recycling, tiered proxy escalation, and anti-bot evasion; a hosted cloud API exists at app.reader.dev.

The interesting bit

The project is refreshingly honest about what production scraping actually requires — TLS fingerprinting, proxy rotation, memory management, graceful degradation — and wraps it all in a one-line ReaderClient constructor. The tiered proxy pools auto-escalate from datacenter to residential when sites push back, which is the kind of detail most scraping tutorials pretend doesn’t exist.

Key highlights

• Stealthed Chrome via CDP: webdriver=false, navigator spoofing, WebRTC masking
• Tiered proxy pools with auto-escalation and health tracking
• Browser pool with configurable size, page limits, and time-based recycling
• Content cleaning strips nav, footers, cookie banners, popups before markdown conversion
• CLI daemon mode keeps the pool warm between requests

Caveats

• Apple Silicon users must manually point CHROME_139_BIN to system Chrome; Hero’s bundled binary lacks arm64 support
• The cloud and self-hosted packages are separate (@vakra-dev/reader-js vs @vakra-dev/reader), which is easy to mix up
• README is heavy on quickstart snippets and light on architecture or failure-mode documentation

Verdict

Worth a look if you’re building AI agents that need reliable web access and you’re tired of duct-taping Puppeteer, proxies, and stealth plugins together. Skip it if your scraping needs are trivial or you need a Python-native stack.