Give your AI a mouse, a sandbox, and a scorecard

What it does This monorepo gives computer-use agents a full habitat. cua-driver runs native macOS and Windows apps in the background so the agent can click and type without commandeering your cursor. cua-sandbox offers a single async Python API to spin up ephemeral Linux, macOS, Windows, or Android boxes locally via QEMU or through the project’s cloud tier. Rounding out the stack, cua-bench evaluates agents on standard benchmarks like OSWorld and Windows Arena, while lume builds near-native Apple-Silicon VMs using Apple’s own Virtualization.Framework.

The interesting bit Most agent frameworks treat the OS as a dumb terminal. Cua treats it as a first-class citizen, virtualizing entire desktops so agents interact with real GUIs instead of REST stubs. The background driver is the subtle hero here: you can actually keep working while your agent fumbles through a spreadsheet in its own invisible session.

Key highlights

• One Sandbox API abstracts Linux containers, macOS VMs, Windows desktops, and even Android devices.
• Background computer-use on macOS and Windows means the agent does not hijack your mouse or keyboard.
• Built-in benchmarks cover OSWorld, ScreenSpot, and Windows Arena, with trajectory export for RL training.
• lume leverages Apple’s Virtualization.Framework for near-native macOS VM performance on Apple Silicon.
• cuabot surfaces sandboxed apps as native-ish desktop windows with H.265 streaming, shared clipboard, and audio.

Caveats

• Linux support for the background driver is still pre-release.
• Optional cua-agent[omni] pulls in ultralytics under AGPL-3.0, which can trip up commercial licensing.
• Several architecture diagrams in the README are currently commented out, suggesting the docs are still catching up to the code.

Verdict Grab it if you are researching or building agents that need to manipulate real operating systems. Pass if you just need a lightweight web-automation wrapper.