Self-hosted AI agents that actually share your GPUs

What it does

HelixML deploys as a microservices stack in your data center or VPC, giving you a web UI and API for building multi-step AI agents. You define agents in a helix.yaml file, and the platform handles RAG over documents, tool orchestration via OpenAPI or MCP, and vision tasks. A built-in GPU scheduler tries to pack models into available VRAM and swap them based on demand, so your hardware doesn’t sit idle while agents pause or resume sessions.

The interesting bit

The project treats agent infrastructure like a cluster scheduler rather than a chat wrapper: it runs on Kubernetes, supports multi-tenancy with Keycloak, and even includes tracing for token-level cost analysis. The license is the notable curveball—free for personal use, education, and small businesses under $10M revenue, but large commercial deployments require a paid license, much like Docker Desktop.

Key highlights

• Multi-provider LLM support (OpenAI, Anthropic, local models) with a unified agent framework
• Built-in RAG pipeline with PGVector, plus document ingestion and web scraping
• GPU runner scheduler that dynamically loads and unloads models to optimize VRAM usage
• Full tracing and observability for agent execution steps, API calls, and token costs
• Microservices architecture (Go backend, React frontend) deployable via Docker Compose or Helm

Caveats

• The license includes a non-compete clause and revenue-based usage restrictions, which may complicate commercial adoption
• Several documentation links in the README resolve to generic top-level URLs, making specific setup guidance hard to locate
• Running the full stack requires managing multiple backing services (Keycloak, PostgreSQL, NATS, GPU runners)

Verdict

Worth evaluating if you’re a platform team needing agent capabilities with strict data residency. Skip it if you want a lightweight library or a framework you can freely repackage into a competing product.