vllm-project/semantic-router
Signal-driven intelligent router for directing requests to appropriate LLM models based on capability, cost, and privacy requirements.

Velocity · 7d
+15
★ / day
Trend
→steady
star history
The system routes incoming requests to optimal models across cloud, data center, and edge deployments. It provides token economics optimization to reduce wasted tokens, LLM safety with jailbreak and PII detection capabilities, and fullmesh coordination across local, private, and frontier models. Built in Go, it integrates with vLLM, Hugging Face Transformers, and supports MCP for agent tooling.