Pocket LLMs: Chat with models that never phone home

What it does

ChatterUI is a mobile frontend for LLMs that lets you run models directly on your Android device or connect to remote APIs. It wraps llama.cpp through a custom React Native adapter (cui-llama.rn) to execute GGUF files locally, and falls back to a broad menu of hosted backends—OpenAI, Claude, Ollama, koboldcpp, and others—when your phone’s RAM can’t handle the load.

The interesting bit

The project treats “local” and “remote” as equally first-class citizens. You can import a model into the app’s storage for faster startup, or point to an external file to avoid duplicating multi-gigabyte weights. The character card system (v2 spec) and per-character chat histories suggest the target audience isn’t just prompt engineers—it’s the roleplay and fiction crowd that wants persistent, structured personas.

Key highlights

• Runs GGUF models on-device via llama.cpp; recommends Q4_0 quantization for Snapdragon 8 Gen 1+ and Exynos 2200+
• Remote mode covers dedicated APIs (OpenAI, Claude, Cohere, Open Router, AI Horde), open-source backends (Ollama, koboldcpp), plus generic completion endpoints
• Custom API templates via a template system for providers not in the default list
• Character Card v2 support with multiple chats per character, sampler customization, and instruct formatting controls
• Integrates with the device’s native TTS engine

Caveats

• Android only; iOS is “currently in development” with no hardware available for testing
• README doesn’t specify minimum Android version or RAM requirements beyond the quantization note
• Local performance depends heavily on your specific chipset and model size—no benchmarks provided

Verdict

Worth a look if you want portable, offline-capable LLM access without building a Termux pipeline. Skip it if you’re on iOS or need desktop-class context windows; this is a phone-first tool with phone-first constraints.