An LLM walks into Minecraft and starts writing its own actions

What it does mindcraft is a JavaScript framework that turns LLMs into Minecraft players, with Mineflayer supplying the body. The bots converse with you, take natural-language instructions, and act on them — mining, crafting, building, exploring. Each bot is defined by a JSON profile (like andy.json) that picks its models, prompts, and example behaviors, and several profiles can share one world. It supports roughly nineteen API providers — OpenAI, Anthropic, Gemini, Groq, DeepSeek, and friends — or can run entirely on local models through Ollama, including the project’s own finetuned andy-4.

The interesting bit The brain is split by role: separate models for chat, code generation, vision, embeddings, and voice synthesis, each independently configurable. The embedding model does quiet, clever work — it semantically picks the few-shot examples most relevant to the current moment instead of dumping every example into the context window. And the task format (goal, starting inventory, timeout, blocked actions) is a benchmark harness in disguise; there’s a companion research effort, MineCollab.

Key highlights

• ~19 API providers supported, plus local inference via Ollama and vLLM — no API key needed if you stay local.
• Per-role model assignment: model, code_model, vision_model, embedding, speak_model.
• Task episodes with goals, inventories, and timeouts — evaluation is built into the product.
• Voice synthesis narrates bot replies (OpenAI and Google only).
• Ships with a Docker setup for containing the riskier code-execution mode.

Caveats

• The bot can write and execute code on your machine. It’s sandboxed but explicitly vulnerable to prompt injection; code-writing is off by default, and even the recommended Docker route “does not guarantee complete safety.”
• Node v24+ may break native dependencies, and macOS installs can hit native-module build errors.
• The bot’s profile name must exactly match its Minecraft account name — otherwise it will spam-talk to itself.

Verdict A rich testbed for embodied LLM agents: researchers, tinkerers, and anyone wanting a Minecraft sidekick will get a lot out of it. If you need a hardened product with a real security story, the README’s own “ye be warned” should settle the question.