Forging unlimited training data for coding agents

What it does SWE-smith is a data factory and training scaffold for SWE-agents. It ingests a GitHub repository, builds a Docker execution environment, and synthesizes task instances—such as program repair or file localization—that are kept only if they break at least one unit test. The result is a dataset of reproducible coding challenges paired with containerized runtimes, which can then be used to fine-tune or perform reinforcement learning on language models.

The interesting bit The project treats open-source repositories as inexhaustible ore: by auto-generating issue text and filtering for test failures, it creates a “SWE-gym” from code that was never originally intended as training material. This shifts the burden from curating benchmark datasets to industrializing data generation.

Key highlights

• Ships with 52,000 pre-generated task instances and 250+ per-repo Docker environments.
• Authors used it to fine-tune Qwen 2.5 Coder into SWE-agent-LM-32B, which scores 40.2% pass@1 on SWE-bench Verified.
• Supports GRPO-style reinforcement learning via integration with SkyRL.
• Accepted as a NeurIPS 2025 Datasets & Benchmarks Spotlight.

Caveats

• Requires Docker and is developed only for Ubuntu 22.04; the authors explicitly do not plan to support Windows or macOS.
• The README’s training example loads the dataset and initializes a container, then leaves the actual loop as a literal TODO: Train! comment—you bring your own trainer.
• Building a dataset from scratch still requires four separate synthesis and filtering stages.

Verdict Researchers and teams building coding agents who need scalable, out-of-the-box training environments should look here; developers seeking a drop-in, fully packaged trainer or cross-platform support should look elsewhere.