A hackable driving simulator with plug-in neural renderers

What it does AlpaSim simulates autonomous vehicles in closed-loop scenarios so researchers can validate algorithms, benchmark regressions, and debug edge cases without physical hardware. It feeds end-to-end driving policies realistic sensor data, vehicle dynamics, and traffic behavior through a modular Python framework built for rapid prototyping.

The interesting bit The simulator treats rendering as a swappable microservice rather than a monolithic engine: the default path uses NVIDIA’s NuRec, but you can substitute OmniDreams via FlashDreams for stateful, video-model-generated imagery aimed at stronger dynamic-object fidelity. The entire stack is horizontally scalable across multiple nodes, which is still rare in research-grade AV tooling.

Key highlights

• Pluggable renderer architecture supporting both traditional (NuRec) and neural video-model (OmniDreams/FlashDreams) backends
• Closed-loop testing with pre-integrated policies including Alpamayo-R1, Alpamayo 1.5, VaVAM, and Transfuser LTFv6
• Microservices design with gRPC interfaces, enabling distributed multi-node deployments and component swapping
• Python-first implementation with extensive configuration, debugging tools, and telemetry via Prometheus
• Apache 2.0 licensed with scene datasets and Hugging Face sample data available

Caveats

• Transfuser LTFv6 support is explicitly marked as "provisional" in the documentation
• OmniDreams neural rendering is supported but requires a separate setup path; the default tutorial uses NuRec
• The built-in policy roster is small and currently NVIDIA-heavy, though community contributions are welcomed

Verdict AV researchers who need a scalable, modular closed-loop testbed for end-to-end policies should look here; skip it if you are after a polished consumer simulator or a zero-configuration drop-in environment.