An inference engine that speaks six SDKs and zero cloud bills

What it does NobodyWho is essentially a set of language bindings around llama.cpp that lets you run GGUF-format LLMs locally on phones, desktops, and inside game engines. It targets Kotlin, Swift, Python, Flutter, React Native, and Godot with a uniform API for offline chat, multimodal input, and tool calling. No API keys, no cloud round-trips.

The interesting bit The engine auto-generates structured grammars from your function signatures for type-safe tool calling, sparing you from hand-written JSON schemas. It also claims to preemptively shift conversation context to retain full memory without hard message limits, which is a tidy workaround for finite context windows.

Key highlights

• GPU-accelerated via Vulkan or Metal across Linux, macOS, Windows, Android, and iOS.
• Supports text, image, and audio inputs.
• Pulls models directly from Hugging Face or any URL.
• Licensed for proprietary commercial use, provided you don’t fork the engine itself closed-source.
• Covers niche platforms like visionOS, watchOS, and the Godot Engine.

Caveats

• Web/HTML5 exports are unsupported for now (see issue #111).
• Mobile OS coverage is uneven: Android and iOS are not available in every binding (e.g., no iOS for Kotlin or Godot).

Verdict Ideal for mobile and game developers who want offline LLM features without managing cloud infrastructure. If you need browser-based inference or a single binding that works on every OS, look elsewhere.