The Infrastructure Layer AI Agents Did Not Know They Needed

The agentic AI boom has produced a glut of models that can see screens and click buttons. Anthropic’s Computer Use API gave Claude a mouse and keyboard in late 2024; OpenAI’s Operator followed weeks later, and by mid-2025 Google’s Project Mariner was offering Gemini subscribers ten concurrent tasks across the web. The open-source ecosystem responded with UI-TARS, Agent S3, Browser Use, and a dozen other perception-planning-action loops. Yet most of these projects share a blind spot: they obsess over the brain and ignore the body. Someone still has to provision the operating system, pipe the screen capture, and isolate the agent so a prompt injection does not format the host machine. That is the gap the CUA project is trying to fill.

CUA is not an agent. It is the substrate beneath the agent. The repository bundles a macOS virtualization layer, a cross-platform background driver, a sandboxing API, a benchmarking harness, and a cooperative desktop wrapper called CuaBot. Think of it as the arena builder in a gladiator movie: the fighters get the glory, but the arena makes the fight possible. The maintainers also curate ACU, a separate repository cataloging research papers and frameworks in the computer-use space, which suggests they see their role as community infrastructure as much as engineering.

The stack is deliberately broad. At the bottom sits Lume, a lightweight macOS and Linux virtualization tool for Apple Silicon that leans on Apple’s Virtualization.Framework to run near-native VMs. Above that, the Cua Sandbox offers a single asynchronous API for spawning ephemeral environments—Linux containers or VMs, macOS, Windows, Android, or bring-your-own images—whether locally via QEMU or on the project’s cloud platform. The sandbox exposes screen capture, shell access, mouse clicks, keyboard input, and even mobile gestures. This uniformity is rarer than it sounds. Most open-source computer-use projects target a single surface: Browser Use and Skyvern focus on the web, Agent S3 targets desktop operating systems but does not abstract the runtime, and Android-specific tools like AgentCPM-GUI stay in their lane. CUA’s API tries to flatten all of them into one interface, so an agent trained on a Linux container can be dropped into a Windows VM without rewriting the control layer.

The Cua Driver adds a critical twist: it can manipulate native desktop applications in the background on macOS and Windows without seizing the user’s cursor or focus. For anyone who has tried to run an automated workflow while actually using their computer, the significance is obvious. The driver exposes a Model Context Protocol server, which means Claude Code, Cursor, Codex, and other MCP clients can treat the machine as a tool without monopolizing it. The project also notes that Linux support for this background backend is currently in pre-release, which is a candid admission that perfect cross-platform parity remains unfinished.

Then there is CuaBot, which wraps a sandbox window into a native-feeling application on the host desktop, complete with H.265 video streaming, shared clipboard, and audio. The effect is co-op computer use: the agent works inside its own isolated window, but the human can watch, intervene, or collaborate without switching contexts. It is a pragmatic answer to the trust problem. Anthropic already recommends virtualized, minimal-privilege environments to mitigate jailbreaking and prompt injection; CuaBot makes that recommendation feel like a feature rather than a chore. The human remains in the loop not by reading logs, but by literally sharing the desktop with a sandboxed colleague.

The benchmarking component, Cua-Bench, reveals the project’s ambition to be infrastructure rather than a toy. It targets OSWorld, ScreenSpot, and Windows Arena—benchmarks that the field increasingly treats as gold standards for measuring whether an agent can actually operate software. By packaging these evaluation suites alongside the runtime, CUA effectively argues that you cannot separate the agent from the measurement apparatus, or the training environment from the test track. The arXiv literature is already moving in this direction: researchers at Shanghai Jiao Tong University recently showed that a mere 312 human-annotated trajectories, augmented via synthesis, could push an open model past Claude 3.7 Sonnet on WindowsAgentArena-V2. That kind of efficient training depends on having reproducible, instrumented environments. CUA wants to be the default place where those trajectories are collected.

Positioning matters here. The proprietary giants—Anthropic, OpenAI, Google—are building agents that operate their clouds or your browser. CUA is building the neutral ground. It supports local QEMU execution, which dovetails neatly with the push toward local inference. A recent survey of the hardware landscape notes that quantized open-weight models like Qwen 3.6 35B can now run on a MacBook Pro with a fraction of their original RAM footprint, retaining nearly all benchmark accuracy. If agents are shrinking enough to run on laptops, they will need laptop-grade sandboxes that do not phone home to a cloud provider. CUA’s local-first matrix—Linux, macOS, Windows, Android—is one of the few open-source attempts to cover that entire surface.

The project is not without rough edges. Linux support for the background driver remains a pre-release backend, and some cloud features, such as bring-your-own images, are marked as coming soon. The breadth of the stack—seven distinct packages spanning virtualization, drivers, SDKs, benchmarks, and a Docker-compatible interface called Lumier—risks fragmentation. There is also the fundamental tension of any infrastructure project: if the agents themselves become good enough to run directly on host OSes with perfect safety, the sandbox layer becomes overhead rather than necessity. For now, though, the field consensus is that agents are not trustworthy enough to run bare. Anthropic’s own documentation notes that scrolling, dragging, and zooming remain difficult for its models, and OpenAI’s Operator still scores only 38% on OSWorld. Agents are clumsy houseguests; CUA is offering the guest house.

Adoption is difficult to quantify precisely—the repository carries a Trendshift badge and a sponsorship program, but no explicit user metrics are listed. Still, the existence of the ACU curated list, which aggregates everything from Anthropic’s API to Large Action Models and reinforcement learning approaches like LOOP, suggests the maintainers are playing a long game as community archivists as well as engineers. They are betting that computer-use agents will proliferate across every operating system and that someone needs to maintain the plumbing.

The open question is whether CUA becomes the de facto standard or merely one option among many. Proprietary clouds will always offer smoother onboarding for non-technical users. Browser-specific frameworks will remain lighter for pure web automation. But for researchers training the next generation of efficient agents, for enterprises that need an agent to fill out a Windows legacy form while a human continues typing emails, or for developers who want to benchmark against OSWorld without cobbling together QEMU and VNC themselves, CUA is positioning itself as the layer that says: here is a computer, ready for whatever mind wants to use it.