Agent vision that lives in the harness, not the model

What it does

This repository supplies command-line vision tools and an agent skill that let text-only models—such as DeepSeek running inside Codex—handle images, read long screenshots, rebuild frontends from sketches, and drive GUI automation. An optional transparent proxy or native plugin intercepts pasted images and built-in image tool calls, so agents like Claude Code, Pi, and OpenCode experience multimodality without native support. All vision processing routes through a standard OpenAI-compatible API that you provide.

The interesting bit

The project treats sight as a harness problem, not a model problem: the bundled skill teaches the agent which tool to invoke, in what sequence, and how to verify the result, effectively encoding visual playbooks into the prompt context. A local proxy layer makes the whole thing feel seamless to the agent, which apparently does not know it is text-only.

Key highlights

• Intent-aware analysis: the tools pass the current task context to the vision API rather than returning generic image descriptions.
• UI restoration playbooks: agents can reconstruct HTML/CSS from screenshots or sketches, with a fast mode aimed at a first visual pass in about three minutes.
• Long-screenshot OCR: stitches scrolling captures by detecting low-content cut bands and merging duplicated overlap.
• Drop-in integrations: single-file native extensions for Pi, Oh My Pi, and OpenCode, plus a proxy for Codex and Claude Code.
• Real-session verification: the authors state the pipeline has been used end-to-end with Codex + DeepSeek, Claude Code, and others.

Caveats

• You must supply your own OpenAI-compatible vision API endpoint and key; the toolkit provides the harness, not the vision model.
• The DeepSeek Harness integration lives in a separate Git submodule, so keeping it current requires manual submodule updates.
• The README claims a properly equipped text-only agent can outperform native multimodal agents, but offers no benchmarks to support that.

Verdict

Useful if you are committed to a text-only model and need your agent to see the screen. If your agent already sees natively, the extra proxy and tool orchestration may be unnecessary.