A QA harness that remembers why your last test broke

What it does

agent-qa is an open-source test harness that lets software teams write end-to-end tests for web and mobile apps in natural language. An agent translates those instructions into actions drawn from visible roles, labels, and screen state. If a click, fill, or select fails because a button moved or a label changed, the harness re-observes the UI and attempts a different path in the same run rather than immediately failing.

The interesting bit

The project treats test execution as a memory problem, not merely a planning problem. Every run builds a persistent memory bank of product, suite, and test observations, and the harness specifically remembers which detours worked during self-healing so future runs can avoid repeating the same mistake. It also caches validated action plans to cut down on redundant planner work and token usage.

Key highlights

• Self-healing execution: re-observes the UI and reroutes mid-test when sub-actions fail, recovering from UI drift and flaky interactions.
• Long-term memory: curates observations and successful healing detours across runs to improve future execution context.
• Sandboxed hooks: runs Node, Bun, Python, or Bash scripts in isolated Docker containers during tests for setup, API calls, fixture seeding, or teardown.
• BYO LLM: works with OpenAI, Anthropic-compatible, Gemini, local models, and subscription services like Codex and Claude Code.
• Human and machine interfaces: includes a dashboard and CLI for developers, plus MCP server support for coding agents.

Verdict

Worth a look if your team is drowning in brittle selectors and wants to write assertions in prose instead of code. Probably overkill if your app has minimal UI churn and your current deterministic tests already run green.