ByteDance’s band-split trick for pulling apart songs

What it does BS-RoFormer is a PyTorch implementation of the ByteDance AI Labs transformer that isolates instruments from mixed audio. It runs axial attention across time and frequency—splitting frequency into bands—so the model can tease apart vocals, drums, or other stems. The repo also includes the follow-up Mel-Band variant and supports stereo inputs and multiple output stems.

The interesting bit The authors found that rotary positional embeddings (RoPE) delivered a huge jump over learned absolute positions, which is the kind of boring-sounding detail that actually moves the SOTA needle. This repo is a clean, hackable reference implementation of that architecture rather than a black-box service.

Key highlights

• Implements both the original Band-Split Roformer and the Mel-Band follow-up.
• Claims to beat the prior first-place music separation score by a large margin (per the paper).
• Supports stereo training and multi-stem output natively.
• Community members have already replicated training and released open weights based on this code.
• Has been used in actual released remixes (Katy Perry, Hatsune Miku), so it’s not just a paper exercise.

Verdict Worth grabbing if you’re researching audio transformers or building source-separation pipelines. If you only need a pretrained drop-in model, the community-trained weights linked in the README are probably a better starting point.