Stop photographing the browser — read its accessibility tree instead

What it does

axuse is an MCP server that lets any agent harness (Codex, Claude Code, OpenCode, Antigravity) drive a browser — and, with platform-specific caveats, a desktop. Instead of the usual computer-use pattern of screenshotting the page and guessing pixel coordinates, it attaches over the Chrome DevTools Protocol and reads the accessibility tree directly. The agent gets a compact snapshot, references elements by stable refs like [ref=e7], and dispatches trusted input events. Each action returns only what changed.

The interesting bit

The whole project is an argument: a screenshot loop is the wrong interface for a machine. Diffed accessibility snapshots cost kilobytes where rasters cost megabytes, element refs survive DOM insertion where coordinates drift, and failures come back as typed errors (REF_NOT_FOUND, STALE_STATE, DIALOG_OPEN) with retry hints instead of silent misclicks discovered three screenshots later. The design is openly distilled from OpenAI’s Codex Chrome extension and the ZCode harness.

Key highlights

• Zero npm dependencies; needs only Node 22+ and a Chromium-family browser
• Server mode boots a private Xvfb so Chrome runs fully headed on display-less servers — no headless fingerprint, with persona coherence (timezone, language, display geometry) and proxy support
• Benchmarked in the README: tied on accuracy with Codex’s Chrome extension and browser-use, but roughly half the median task time and half the tokens
• CAPTCHA solvers are deliberately out of core — commercially licensed plugins hook in through a frozen seam, with a free reference plugin in examples/
• Desktop control spans verified Windows, opt-in macOS (TCC permission checks), and X11/XWayland Linux with no extra packages

Caveats

• The benchmark is self-reported, small (16 tasks, plus a 7-task extension run once), and run by the project’s own consultancy — treat the numbers as directional, not gospel
• Core is open, but the actual CAPTCHA solvers are paid add-ons; the free repo ships none
• The README is dense with environment variables and modes; the operational surface is wider than “zero dependencies” suggests

Verdict

If you’re wiring an agent harness to a browser and are tired of vision-token bills and misclicks, this is one of the more principled takes on the problem. If you need CAPTCHA solving out of the box or a non-Chromium browser, look elsewhere.