Serverless GPU inference you can actually host yourself

What it does

Beta9 is the open-source core of Beam. It wraps container orchestration—GPU scheduling, autoscaling, and distributed storage—behind a Pythonic interface so you write functions instead of infrastructure code. The runtime handles fan-out to hundreds of containers, scale-to-zero, and hot-reloading for inference endpoints, sandboxes, and background tasks.

The interesting bit

The project ships a custom container runtime that launches containers in under a second, which is the difference between “serverless” and “serverless but make me wait.” It also blurs the line between managed cloud and open-source: the same AGPL engine powers Beam’s commercial offering, so self-hosting is not a second-class fork.

Key highlights

• GPU support on your own hardware or Beam’s cloud, including 4090s and H100s
• Autoscales endpoints and task queues to hundreds of containers, then scales to zero
• Claims sub-second container launches via a custom runtime
• Isolated sandboxes for running LLM-generated code safely
• Licensed under AGPL, with a fully managed cloud option hosted by Beam

Verdict

Worth a look if you want serverless GPU abstraction without cloud lock-in; skip it if you aren’t running inference, sandboxes, or heavy background jobs.