Teaching neural networks to jam in five-part harmony

What it does MuseGAN trains generative adversarial networks on the Lakh Pianoroll Dataset to produce pop-song phrases with five separate instrument tracks. The system can compose from nothing or fill in missing parts around a track you supply, outputting pianoroll arrays, images, or MIDI-ready files.

The interesting bit The latest implementation trades control for efficiency: it borrows 3D convolutions from the BinaryMuseGAN architecture, which shrinks the network but sacrifices the ability to feed different latent variables to individual measures or tracks. The original AAAI 2018 experiments remain available in the v1 folder.

Key highlights

• Generates polyphonic pop phrases for bass, drums, guitar, piano, and strings simultaneously
• Supports both unconditional generation and track-conditional accompaniment
• Uses 3D convolutional layers for temporal structure, yielding a smaller model footprint
• Exports numpy arrays, PNG visualizations, and compressed pianorolls convertible to MIDI via pypianoroll
• Original research code preserved in v1; a PyTorch reimplementation lives in a separate repository

Caveats

• The current 3D-convolution design explicitly limits fine-grained control over individual measures or tracks compared with earlier versions
• This is the original research codebase; users seeking a PyTorch implementation are pointed to an external tutorial repository

Verdict A solid reference if you study symbolic music generation or need a working multi-track GAN baseline. Look elsewhere if you want granular control over every instrument line or a modern PyTorch-native workflow.