Spotify's lightweight audio-to-MIDI converter runs almost anywhere

What it does

Basic Pitch is an automatic music transcription library from Spotify’s Audio Intelligence Lab. Feed it an audio file and it spits out a MIDI transcription, including pitch bends. It is instrument-agnostic and handles polyphonic input, though it works best on a single instrument at a time.

The interesting bit

The model punches above its weight: its note accuracy and multipitch support compete with much larger, resource-hungry transcription systems. It also ships with multiple runtime flavors—TensorFlow, CoreML, TensorFlowLite, and ONNX—so it defaults to the least burdensome option for your OS rather than dragging in the full TF stack.

Key highlights

• Instrument-agnostic polyphonic transcription with pitch bend detection
• Cross-platform with OS-optimized default runtimes (CoreML on macOS, TensorFlowLite on Linux, ONNX on Windows)
• Sibling TypeScript repo and a third-party VST plugin (NeuralNote) extend its reach beyond Python scripts
• Accepts common audio formats via librosa, resampling anything to 22050 Hz internally
• Backed by an ICASSP 2022 paper, though the authors note the code has since improved beyond the published results

Caveats

• macOS M1 support is currently limited to Python 3.10; the README suggests a virtual machine for other versions
• Stereo inputs are down-mixed to mono before analysis
• Very large files may exhaust disk space during processing unless you stream the audio in windowed segments

Verdict

Musicians, producers, and developers who need quick, offline audio-to-MIDI conversion for single-instrument sources should take a look. If you need full orchestral transcription or deep DAW integration out of the box, this is a starting point rather than a finished pipeline.