Local LLMs on every Apple OS, plus a macro for types

What it does

LLM.swift is a Swift abstraction over llama.cpp that loads and runs GGUF models locally on macOS, iOS, watchOS, tvOS, and visionOS. You can bundle a model with your app or pull one directly from Hugging Face at runtime. The library handles chat templating, history rotation, and streaming responses through an open Swift class meant to be subclassed.

The interesting bit

The standout feature is @Generatable, a Swift macro that turns structs and enums into JSON schemas to constrain the model’s output into valid, type-safe objects. The project is essentially well-crafted glue code over a battle-tested C++ engine, but the macro pushes it beyond a thin wrapper into something that actually feels native to Swift’s type system.

Key highlights

• Runs inference entirely offline on any Apple platform using standard GGUF models.
• Downloads compatible models directly from Hugging Face with progress callbacks.
• @Generatable macro produces structured output for structs, enums, arrays, and nested types.
• Built-in chat templates (ChatML, Gemma, etc.) and conversation history with preprocess, postprocess, and update hooks.
• The core LLM class is open for subclassing when you need fine-grained control over the generation loop.

Caveats

• Mobile memory is tight; the README explicitly warns that a poorly chosen maxTokenCount can silently truncate conversation context.

Verdict

Swift developers building Apple-native apps that need private, offline LLM inference should look here; if you need cross-platform support or already have a llama.cpp pipeline, it adds little.