What it does Off Grid is a mobile AI suite that downloads and runs models locally on Android, iOS, and macOS. It handles text generation, Stable Diffusion image creation, vision analysis, Whisper speech-to-text, document RAG, and tool calling — all without network access. You can also fall back to local-network OpenAI-compatible servers (Ollama, LM Studio) if you want bigger models without the cloud.

The interesting bit The project bundles a surprising amount of native plumbing: llama.cpp and whisper.cpp via React Native bindings, Core ML diffusion on iOS, NPU acceleration on Snapdragon, on-device MiniLM embeddings for knowledge base search, and a full tool-calling loop with runaway prevention. The “Pro” tier is being crowdfunded with a refreshingly small ask — $10 from 100 people to fund 12 weeks of full-time work.

Key highlights

• Runs GGUF models (Qwen 3, Llama 3.2, Gemma 3, Phi-4, custom) at 15-30 tok/s on flagship devices
• Stable Diffusion with real-time preview: 5-10s on NPU, ~15s on CPU
• Vision inference (~7s) with SmolVLM, Qwen3-VL, Gemma 3n
• On-device RAG: chunking, MiniLM embeddings, SQLite storage, cosine similarity retrieval
• Tool calling with built-in search, calculator, datetime, device info, knowledge base
• Cross-platform test coverage: Jest/RNTL, JUnit, XCTest, plus Maestro E2E on BrowserStack

Caveats

• Performance drops significantly on mid-range hardware (5-15 tok/s, ~30s image gen)
• macOS support is via Mac Catalyst/iPad compatibility, not a native Mac app
• The “Pro” features (custom personas, Kokoro TTS, MCP servers, larger models) are pre-orders shipping in 12 weeks, with refund promise but no working code yet

Verdict Worth a look if you want a reference implementation for running multiple AI modalities on-device, or if you actually need offline AI on your phone. Skip if you’re expecting ChatGPT-level quality from a 3B parameter model — the tradeoffs are real and documented.