OpenCoder-llm/OpenCoder-llm
OpenCoder open-sources the full code LLM recipe
Most code LLMs hand you a black box; OpenCoder hands you the model weights, training data, cleaning pipeline, and intermediate checkpoints.
Not currently ranked — collecting fresh signals.
star history
What it does
OpenCoder is a family of code-focused language models—1.5B and 8B parameters, base and instruct variants—trained from scratch on 2.5 trillion tokens of raw code and code-related web data. The project releases not just model weights but also the pre-training and fine-tuning datasets, a data-cleaning pipeline, intermediate checkpoints, and an evaluation framework called OpenCodeEval. It supports both English and Chinese.
The interesting bit
The breadth of the release is the story: instead of a weights-only drop, the team published the opc_data_filtering pipeline, over 4.5 million supervised fine-tuning examples, annealing corpora, and per-stage checkpoints so outsiders can audit the training trajectory. They appear to treat reproducibility as a first-class feature.
Key highlights
- 1.5B and 8B base and instruct models with 4K and 8K context lengths.
- Pre-training corpus mixes 90% raw code with 10% code-related web data; post-training includes 4.21M stage-one and 375K stage-two SFT examples.
- Released artifacts include the
opc_data_filteringcleaning pipeline,OpenCodeEvalbenchmark harness, and intermediate base checkpoints. - Bilingual support for English and Chinese.
Caveats
- Some datasets and artifacts are still being organized and uploaded progressively.
- The README mentions both that inference code is provided and that it is “forthcoming,” so the exact status of inference utilities is unclear.
Verdict
Researchers who want to train or fine-tune their own code models should care—this is essentially a turnkey recipe. If you just need a drop-in coding assistant via API, the release is overkill.
Frequently asked
- What is OpenCoder-llm/OpenCoder-llm?
- Most code LLMs hand you a black box; OpenCoder hands you the model weights, training data, cleaning pipeline, and intermediate checkpoints.
- Is OpenCoder-llm open source?
- Yes — OpenCoder-llm/OpenCoder-llm is open source, released under the MIT license.
- What language is OpenCoder-llm written in?
- OpenCoder-llm/OpenCoder-llm is primarily written in Python.
- How popular is OpenCoder-llm?
- OpenCoder-llm/OpenCoder-llm has 2.1k stars on GitHub.
- Where can I find OpenCoder-llm?
- OpenCoder-llm/OpenCoder-llm is on GitHub at https://github.com/OpenCoder-llm/OpenCoder-llm.