A neural audio compressor you can actually play live

What it does

RAVE is a variational autoencoder for audio from IRCAM’s ACIDS team. It learns compressed latent representations of sound, then reconstructs or transforms them fast enough for live performance. The project ships as a Python training pipeline and exports to TorchScript models that load into a VST plug-in or the nn~ external for Max/MSP and PureData.

The interesting bit

The twist is that RAVE is built for the stage, not just the lab. It offers causal convolution modes for low latency, a raspberry config that runs realtime on a Pi 4, and v3 adds Adaptive Instance Normalization so you can perform style transfer between source and target audio inside a Max/MSP or PureData patch.

Key highlights

• Eight model flavors from lightweight raspberry (5 GB VRAM) to a StyleGAN-inspired v3 (32 GB), plus discrete SoundStream-like codecs.
• Realtime manipulation of the latent space via nn~ in Max/MSP or PureData, or through a beta VST.
• Data-augmentation presets (mute, compress, gain) for training on smaller corpora.
• Streaming export mode with cached convolutions; forgetting it reportedly causes audible clicking artifacts in live setups.
• Pretrained streaming models are available for download.

Caveats

• The VST and some features (GPU acceleration in nn~, the v2_nopqmf config) are marked as experimental or beta.
• Lazy preprocessing avoids disk bloat but spikes CPU load heavily, especially on Windows.
• Training requires careful dataset sizing: too few batches or overly short audio files will silently stall preprocessing or crash the internal latent PCA.

Verdict

Electronic musicians, sound-installation artists, and researchers who need a fast neural audio codec should look here. If you only need offline batch audio generation and don’t care about millisecond-latency reconstruction, simpler waveform-to-waveform tools may be less fussy.