LLMs that never leave the phone: Flutter gets on-device AI

What it does

flutter_edge_ai brings local LLM inference to Flutter apps on Android, iOS, Web, macOS, Windows, and Linux. It supports text generation with Gemma and other models, plus multimodal input, function calling, embeddings, RAG, and speech-to-text/text-to-speech. The underlying inference rides on MediaPipe and LiteRT (formerly TFLite), with ONNX in the mix as well.

The interesting bit

The genuinely unusual part is the platform breadth — most on-device LLM tooling picks a favorite OS and stops there, while this targets all six Flutter targets including Web. Running a local model in a browser tab is not something you see every day.

Key highlights

• Text generation with Gemma and other LLMs, fully on-device
• Multimodal support (text plus other input types)
• Function calling, embeddings, and RAG pipelines
• Speech-to-text and text-to-speech
• Cross-platform: Android, iOS, Web, macOS, Windows, Linux

Caveats

• The repo’s language is listed as JavaScript despite being a Dart/Flutter package — likely build tooling, but worth noting the codebase composition is unclear from the description alone.
• Model size and performance characteristics per platform aren’t stated in the provided description, so expect to verify on your target hardware.

Verdict

If you’re building a Flutter app where privacy or offline operation matters — or you just want a chatbot that works in airplane mode — this is worth a look. If you need frontier-model quality, on-device Gemma won’t substitute for a hosted API."}