Open-source coder beats GPT-4 Turbo, brings its own package manager

What it does AutoCoder is a family of fine-tuned code generation models—33B and 6.7B variants built on DeepSeek-Coder, plus a newer 7B release built on CodeQwen1.5—that generate Python. The project reports a 90.9% pass rate on the HumanEval base dataset, edging out the 90.2% reported for GPT-4 Turbo (April 2024). It also includes a code interpreter that executes on demand and automatically installs any missing Python packages before running.

The interesting bit Unlike OpenCodeInterpreter, which executes every snippet it generates, AutoCoder’s interpreter fires only on demand—behavior the README compares to GPT-4 Turbo. The twist is that AutoCoder can reach out and install external libraries on the fly, something the authors claim GPT-4o cannot do.

Key highlights

• Claims 90.9% on HumanEval base and 82.5% on MBPP base, with extra-test scores of 78.0% and 70.6% respectively.
• Ships in three sizes: 33B, 6.7B, and a 7B variant; all are fine-tunes of existing open models rather than trained from scratch.
• Interpreter runs only when the user requests it, not on every generation.
• Bundles a Gradio web demo that exposes the interpreter for interactive use.
• Academic paper available on arXiv describing the AIEV-Instruct fine-tuning approach.

Caveats

• Reproducing the headline HumanEval and MBPP numbers requires greedy decoding and post-processing with EvalPlus sanitization; the README warns that enabling sampling gives different results.
• The authors recommend Linux for deployment.
• DS-1000 accuracy figures are absent from the README; only the evaluation script is provided.

Verdict Worth a look if you want an open-weight coding model that runs locally and handles its own dependency installation during execution. Pass if you need a fully general coding agent or expect leaderboard numbers to match out of the box without fiddling decoding settings.