ByteDance Packs a Terminal Agent and Desktop Operator into One Repo

What it does This repository ships two related automation tools built around the UI-TARS vision model. Agent TARS is a CLI and web-based agent stack for controlling browsers and computers via natural language. UI-TARS Desktop is a native application that performs local or remote computer and browser operations through a graphical interface. Both interpret on-screen visuals to decide actions rather than relying purely on APIs or HTML parsing.

The interesting bit The project refuses to pick a single interface paradigm. Agent TARS is built for terminal and server workflows with an event-stream architecture, while UI-TARS Desktop is a point-and-click native app for controlling remote machines. Underneath, both lean on the same pixel-level visual grounding to drive cursors and issue keystrokes.

Key highlights

• Agent TARS runs in both headful Web UI and headless server modes
• Browser control supports visual grounding, DOM parsing, or a hybrid of the two
• The kernel uses the Model Context Protocol (MCP) and can mount external MCP servers
• UI-TARS Desktop provides free local and remote computer and browser operators
• A cross-platform SDK is available for building custom GUI automation agents

Verdict Worth exploring if you want an open-source agent that manipulates live interfaces through vision rather than brittle selectors; less useful if you need a minimal, single-purpose library instead of a full stack.