A pocket-sized LLM server with no chaperone

What it does This is a mobile-first chat app built on Flutter and llama.cpp that runs GGUF models directly on Android or iOS. No accounts, no API keys, no cloud — conversations and inference both stay on the device. It ships with a model library (download or import GGUF files in-app), persistent chat history via Hive, and live tokens/sec metrics.

The interesting bit Beyond chat, it exposes a built-in OpenAI-compatible REST server on 127.0.0.1:4891/v1, so your phone’s model can back any standard client, script, or IDE extension. Your phone becomes a private, offline inference endpoint — which is a genuinely unusual deployment target.

Key highlights

• Runs abliterated/uncensored models with no refusals or safety filters
• Fully offline after model download; nothing leaves the device
• OpenAI-compatible local API with streaming support
• Recommended models span ~1.6 GB (Gemma 2 2B) to ~5.3 GB (Gemma E4B Heretic)
• One Dart codebase targeting Android, iOS, Windows, macOS, and Linux

Caveats

• Desktop builds compile but are explicitly unpolished — the maintainers are actively recruiting testers
• iOS is a pre-release .ipa requiring sideloading via TrollStore or AltStore
• The x86_64 APK is still pinned to v1.0.0 while others ship v2.0.0

Verdict Worth a look if you want private, offline LLM chat or a pocket inference server and have the RAM for it. If you need polished desktop apps or curated, filtered models, this isn’t that — by design.