Teaching an LLM to edit the code that defines itself

What it does

Darwin Gödel Machine (DGM) treats its own Python codebase as a mutable target. An LLM-based coding agent proposes edits to its own source files, then empirically validates whether the mutated version performs better on SWE-bench and Polyglot tasks. The loop repeats, theoretically bootstrapping increasingly capable self-modification.

The interesting bit

This is a practical, benchmark-grounded take on the Gödel Machine concept: the agent empirically validates each self-modification using benchmark scores rather than formal proofs. The codebase is simultaneously the product and the optimization target.

Key highlights

• Self-modification loop: the agent edits coding_agent.py and related files, then tests the offspring
• Validation grounded in SWE-bench and Polyglot benchmark scores
• Full experiment logs from foundation model outputs are published to Google Drive
• Explicit safety warning: executes untrusted, model-generated code inside a sandbox
• Built by Sakana AI; accompanied by an arXiv paper and blog post

Caveats

• Depends on commercial LLM APIs (OpenAI and Anthropic) and a container sandbox; not a fully local or self-contained system
• README offers file structure and setup but stays vague on quantitative results or convergence behavior (those live in external Drive logs)
• Running it means letting an LLM generate and execute arbitrary code on your machine, which the authors flag as a genuine safety risk

Verdict

Worth a look if you study recursive self-improvement, autopoietic systems, or LLM-driven meta-learning. Skip it if you want a stable, production-ready coding assistant rather than a research sandbox.