Giving LLMs the keys to your Android accessibility layer

What it does

OpenDroid is an Android agent that turns natural-language commands into actual phone actions. Ask it to text a contact, set an alarm, and check the weather, and it will plan the steps, launch the relevant apps, execute actions via the AccessibilityService API, and replan if a step fails. It feeds screenshots to vision-capable LLMs when available and falls back to reading the accessibility tree on older devices.

The interesting bit

Instead of begging apps for private APIs, OpenDroid treats the entire Android UI as a generic automation surface. It pairs that approach with a four-tier memory system—working, episodic, semantic, and procedural—and a fallback chain across 12 LLM providers so a dead API key doesn’t kill your automation.

Key highlights

• Executes 60+ actions across system settings, messaging, navigation, media, and finance by driving the UI layer through the accessibility service.
• Bundles a local model manager that downloads LiteRT-LM models in the background, verifies SHA-256 hashes, and gates access behind EncryptedSharedPreferences.
• Supports 12 LLM backends including Gemini, Claude, OpenAI, Groq, DeepSeek, and Ollama, with automatic provider failover.
• Runs a multi-tier memory stack and offline wake-word detection for hands-free voice control.
• Built in Kotlin with Jetpack Compose, Dagger-Hilt, and Room, targeting Android 15.

Verdict

Grab it if you want a hackable, open-source phone assistant that actually controls the UI for you. Avoid it if you balk at granting broad accessibility and notification permissions to a project maintained by a solo developer.