Real-time music generation that runs inside your DAW, offline

What it does Magenta RealTime 2 (MRT2) is an open-weights model and inference stack for generating music in real time. It ships as a Python library with JAX and MLX backends, a C++ streaming engine optimized for Apple Silicon, and example apps including an AUv3 plugin and a standalone macOS app. You can prompt it with text like disco funk and get back audio faster than playback speed—at least if your Mac is beefy enough.

The interesting bit The project draws a hard line between real-time streaming and offline inference. The 230M-parameter mrt2_small model streams on any Apple Silicon Mac, while the 2.4B-parameter mrt2_base model demands a Pro Max chip to keep up live; anything weaker falls back to non-real-time generation. That honesty about hardware tiers is refreshing in a space that usually pretends latency does not exist.

Key highlights

• Open-weights model (230M and 2.4B parameters) hosted on Hugging Face
• Dual Python backends: JAX for flexibility, MLX for Apple Silicon efficiency
• C++ inference engine (magentart::core) purpose-built for low-latency streaming
• Ready-made AUv3 plugin and standalone macOS app examples
• Offline inference also supported on NVIDIA GPUs via the Python library
• Supervised fine-tuning support is on the roadmap

Caveats

• Real-time streaming is strictly Apple Silicon territory; the compatibility matrix only covers M-series Macs
• The 2.4B-parameter model requires a Pro Max chip for live generation, so the full-quality experience is hardware-gated
• The project is currently inference-only; supervised fine-tuning is listed as a future update

Verdict Worth exploring if you are an audio developer or researcher who wants an open, local alternative to cloud music APIs and can live inside the Apple ecosystem. Windows producers looking for a drop-in real-time plugin should look elsewhere—at least for now.