← all repositories

vllm-project/semantic-router

Signal-driven intelligent router for directing requests to appropriate LLM models based on capability, cost, and privacy requirements.

4.3k stars Go LLMOps · EvalAgents
semantic-router
Velocity · 7d
+15
★ / day
Trend
steady
star history

The system routes incoming requests to optimal models across cloud, data center, and edge deployments. It provides token economics optimization to reduce wasted tokens, LLM safety with jailbreak and PII detection capabilities, and fullmesh coordination across local, private, and frontier models. Built in Go, it integrates with vLLM, Hugging Face Transformers, and supports MCP for agent tooling.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.