Give Your Android Phone a Backend and a Job

What it does

OpenGUI is a backend-and-client system that lets an AI operate real Android apps by reading the screen, planning steps, and firing actions through the phone’s accessibility service. It returns structured summaries after the run, and the backend keeps task state alive so workflows can stretch across hours instead of collapsing after a single session.

The interesting bit

Most phone agents are fragile demo loops; OpenGUI treats the phone as a networked worker with persistent standby sockets and a split-brain model setup. You can route planning to cheap text models and vision to a VLM, and dispatch jobs from Discord or Telegram while the device sits in a drawer.

Key highlights

• Graph-based orchestration with distinct Plan Supervisor, Executor Graph, and Summarizer stages
• Model routing supports mixing providers—e.g., Qwen for planning and Doubao for vision—to cut costs by roughly 10× to 15× versus an all-Opus setup
• Standby gateway keeps Android devices connected via persistent sockets for remote task dispatch
• Built-in IM and REST entry points: Discord, Telegram, Feishu, or local CLI
• Designed for long-running tasks with review and retry loops managed server-side

Caveats

• Requires a real Android device or emulator, USB debugging, and AccessibilityService permissions
• Not an always-on assistant; tasks are manually triggered or fed through configured dispatch channels
• The authors warn that long-running reliability still needs real-world testing and that ready-to-run examples are currently sparse

Verdict

Pick this up if you need a server-side Android agent that plans and recovers rather than blindly tapping through UI. Pass if you want a zero-setup phone app that just works out of the box.