bentoml/Yatai
Kubernetes operator for ML models that speaks GitOps
Yatai turns BentoML model packages into Kubernetes-native deployments via custom resources, so DevOps teams don't need to hand-craft containers for every model update.
Velocity · 7d
+0.5
★ / day
Trend
→steady
star history
What it does
Yatai is the Kubernetes deployment operator for BentoML, an ML model packaging framework. It installs into a cluster and adds three CRDs: BentoRequest, Bento, and BentoDeployment. You push a packaged model (a “Bento”) to Yatai’s registry, then apply a YAML manifest to deploy it with autoscaling, resource limits, and ingress — standard kubectl workflow, no custom CLI required for deployment.
The interesting bit
The project splits into three installable components — core Yatai, image builder, and deployment controller — which lets teams adopt only what they need. The BentoDeployment CRD is the hook: it treats an ML service like any other Kubernetes workload, meaning existing GitOps pipelines and RBAC policies apply without special-casing model deployments.
Key highlights
- CRD-native deployments: Define model services in YAML, version them in git, deploy with
kubectl apply - Per-component installation: Core registry, image builder, and deployment controller install separately via Helm/scripts
- Autoscaling and resource controls: HPA-style
minReplicas/maxReplicasplus CPU/memory limits at both deployment and runner level - Web UI + API tokens: Browser-based setup and token management for pushing Bentos from the CLI
- Elastic License 2.0: Not open source; source-available with usage restrictions
Caveats
- Under construction for BentoML 1.2: The README warns that Yatai for the current BentoML release is being rebuilt; a “Yatai 2.0 Proposal” is open for comment
- Quick-install scripts are dev-only: Production requires manual Helm installation; the one-liner scripts bundle PostgreSQL and MinIO for testing
- Three separate installs for full functionality: The quick start requires running three different bash scripts to get registry, builder, and deployment features
Verdict
Worth evaluating if you’re already using BentoML and want Kubernetes deployments without building a custom bridge. Skip if you need a fully open-source stack or aren’t committed to the BentoML packaging format — the ELv2 license and version-rewrite uncertainty add friction for new adopters.