Five-Minute AI Research Sprints on Apple Silicon

What it does

This is an Apple Silicon-native port of Andrej Karpathy’s autoresearch framework, replacing PyTorch with Apple’s MLX. A coding agent follows a protocol in program.md, edits a single train.py file, and runs a fixed five-minute training experiment. If val_bpb improves, the change stays; otherwise git reverts it. The loop repeats, automating architecture and hyperparameter search entirely on your Mac.

The interesting bit

Because the budget is wall-clock time rather than step count, smaller models often outperform larger ones on Apple Silicon simply by squeezing in more updates. The project’s overnight runs show a Mac Mini and an M4 Max discovered genuinely different winning recipes—evidence that local hardware quirks are worth studying, not just ignoring.

Key highlights

• Native MLX training with unified memory; no PyTorch or CUDA installation needed
• Rigid keep-or-revert loop driven by one metric (val_bpb) and one mutable file
• Baseline dropped from 2.667 to 1.808 val_bpb mostly by reducing depth from 8 to 4 layers
• Long runs found divergent optimal stacks across M4 Max and Mac Mini
• MFU reporting is a placeholder; no Apple Silicon FLOPs reference exists yet

Caveats

• The public branch sticks to AdamW; the Muon optimizer explored in the Mac Mini run lives in a non-default branch
• Expect 6–7 minutes per full cycle, not the bare five-minute training window
• Evaluation uses a smaller token budget than upstream to prioritize iteration speed

Verdict

A fun sandbox for Mac-owning ML hackers who want to watch an agent discover hardware-specific training tricks. Skip it if you need datacenter-scale baselines or rigorous FLOPs accounting.