← all repositories
dineshsoudagar/local-llms-on-android

Your phone is the home AI server now

A Kotlin app that runs Gemma, Qwen, and DeepSeek locally on Android — and then serves them to your whole LAN with an OpenAI-compatible API.

local-llms-on-android
Too new to rank. Velocity needs a few days of history.
not enough history yet
star history

What it does Pocket LLM runs small language models (Gemma 4, Qwen3, DeepSeek R1 distills, and more) entirely on your phone via LiteRT or ONNX Runtime. Beyond chat, it handles PDFs with OCR, images, audio transcription with offline Whisper, and dictation — all on-device, no telemetry. The twist: it can also act as a LAN server, exposing the phone’s model to other devices through a web UI and OpenAI-compatible endpoints.

The interesting bit Turning a phone into a shared inference endpoint for your home network is the genuinely unusual part — the README even suggests using it as a backend for coding agents like OpenCode, with native tool calls from supported LiteRT models. It also fits context size to your phone’s actual memory and learns from native crashes, which is a pragmatic touch most desktop LLM tools skip.

Key highlights

  • Fully offline after model download: chat, OCR, camera, image input, and inference stay on-device
  • Model lineup spans Qwen3 0.6B for low-end phones up to Gemma 4 E4B for flagships
  • Auto-compact summarizes older messages instead of dropping them when chats get long
  • LAN server with web UI, chat history, and OpenAI-compatible model/chat-completion endpoints
  • Import your own .litertlm models (beta), with per-device detection of image/audio support

Caveats

  • Custom-model import is explicitly experimental — larger models or contexts may fail to load or crash, and native tool calling isn’t exposed for them
  • The GitHub APK (v1.7.0) and Google Play version (v1.7.5) differ: GGUF support, web search, and image tools are Play-only, some behind a €2.29–2.99 one-time Pro tier
  • Video input, generated audio, and custom thinking controls are not enabled

Verdict Worth a look if you want private, offline LLM chat on Android or a cheap always-on inference box from an old phone. Skip it if you need large models or rock-solid custom-model support — the small-model ceiling is real.

Frequently asked

What is dineshsoudagar/local-llms-on-android?
A Kotlin app that runs Gemma, Qwen, and DeepSeek locally on Android — and then serves them to your whole LAN with an OpenAI-compatible API.
Is local-llms-on-android open source?
Yes — dineshsoudagar/local-llms-on-android is open source, released under the MIT license.
What language is local-llms-on-android written in?
dineshsoudagar/local-llms-on-android is primarily written in Kotlin.
How popular is local-llms-on-android?
dineshsoudagar/local-llms-on-android has 502 stars on GitHub.
Where can I find local-llms-on-android?
dineshsoudagar/local-llms-on-android is on GitHub at https://github.com/dineshsoudagar/local-llms-on-android.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.