Training a thinking LLM from scratch on pocket change

What it does

Cortex is an end-to-end training suite for a diminutive Mixture-of-Experts language model. It walks through five sequential stages—Pretrain, Midtrain, SFT, DPO, and PPO—with manual checkpoint shuffling between each phase. The resulting model can be served locally through app.py and supports a reasoning toggle via /think and /no think tags.

The interesting bit

The architecture is aggressively frugal: 0.1B total parameters with only ~67M active during inference, letting the full lifecycle run on a modest quad-MLU370 rig of domestic chips. For the RLHF stage, the project outsources reward modeling to an external LLM-as-a-Judge API rather than training its own critic, which keeps the codebase small but leaves the final alignment step dependent on third-party cloud credits.

Key highlights

• Complete five-stage pipeline: Pretrain → Midtrain (context stretch to 2048) → SFT → DPO → PPO.
• MoE design keeps active compute low; targets high throughput on resource-constrained devices.
• First README-claimed full-stack training flow adapted for domestic MLU370 silicon.
• Optional Attention Residuals mechanism gated behind an ENABLE_ATTN_RES flag in utils.py.
• Model definitions and training logic live in separate repos to keep concerns decoupled.

Caveats

• PPO training requires an external API key for the judge LLM (defaulting to SiliconFlow), so the RL stage is not fully self-contained.
• Hardware requirements beyond the MLU370 mention are unspecified; whether this fits on a single consumer GPU is unclear.
• Stage-to-stage checkpoint conversion is manual—DeepSpeed outputs must be converted, renamed, and shuffled between directories before the next script will run.

Verdict

A solid educational sandbox if you want to touch every phase of LLM creation without a corporate cloud budget. Not the place to look if you need a foundation model that can actually replace GPT-4.