A Rust AI assistant that dreams in 3D

What it does

LocalGPT ships as two Rust crates in one repo. localgpt is a local-first AI assistant with persistent markdown memory, autonomous task queues, and a daemon mode with HTTP API. localgpt-gen turns natural language into explorable 3D scenes using the Bevy engine — geometry, PBR materials, lighting, audio, and scripted behaviors, exported to glTF or HTML.

The interesting bit

The memory system is deliberately low-tech and therefore inspectable: SQLite FTS5 plus sqlite-vec for semantic search, with MEMORY.md, HEARTBEAT.md, and SOUL.md sitting in plain text on disk. The world generator learns your style across sessions and can run headless for overnight batch experiments or CI pipelines. There’s even an MCP server so Claude Desktop can call it as a tool.

Key highlights

• Single static binary, no Node.js/Python/Docker runtime required
• Kernel-enforced sandboxing (Landlock/seccomp on Linux, Seatbelt on macOS) with signed policy files
• Supports local models via LM Studio/Ollama and cloud providers (Anthropic, OpenAI, xAI, etc.)
• Headless world generation with style hints and experiment queuing
• glTF/GLB export plus browser-viewable HTML output

Caveats

• The README mentions security features but doesn’t quantify their coverage or provide audit results
• “Built in 4 nights” per the blog — impressive velocity, but implies immaturity in some subsystems

Verdict

Grab it if you want a hackable, fully local AI stack with an unusual 3D generation sidecar. Skip if you need battle-tested enterprise guardrails or deep collaborative editing in generated worlds.