Teaching Transformers to Write Classical Scores in ABC

What it does NotaGen is a transformer that writes classical sheet music in ABC notation after being trained on 1.6 million pieces. You feed it a prompt like Romantic-Schubert-Piano and it spits out notation renderable as PDF or audio. The project ships pre-trained weights up to 516M parameters, plus a newer NotaGen-X variant that went through extra rounds of post-training and reinforcement learning.

The interesting bit Instead of treating music generation as a special snowflake, the team applied the standard LLM lifecycle: pre-training, fine-tuning on ~9K classical works, and reinforcement learning. Their twist is CLaMP-DPO, which uses the CLaMP 2 music-text model to automatically score generated pieces and drive preference optimization—no human annotators or hand-tuned reward functions required.

Key highlights

• Three model sizes (110M, 244M, 516M) with checkpoints for pre-training, fine-tuning, and RL stages on HuggingFace
• NotaGen-X iterates the pipeline with an added post-training stage and repeated RL, borrowing ideas from DeepSeek-R1
• Outputs ABC notation convertible to MusicXML; available as a web demo, HuggingFace Space, Colab notebook, and ComfyUI node
• CLaMP-DPO relies solely on the CLaMP 2 contrastive model for reward signal, avoiding manual reward engineering
• NotaGen-X inference needs about 8GB of GPU memory; training the large base model demands 24GB

Caveats

• Pre-trained weights alone do not support conditional period-composer-instrumentation generation; you must use the fine-tuned or RL checkpoints
• Training your own model is not a weekend project: it requires downloading external CLaMP 2 and M3 weights, editing multiple config files, and managing ABC-to-MusicXML preprocessing pipelines

Verdict A solid pick for researchers or musicians who want controllable, symbolic classical generation. If you just need a quick lo-fi beat or wav file, this is the wrong aisle—NotaGen thinks in staves and measures, not waveforms.