vllm-project/guidellm
A benchmarking platform that evaluates LLM performance under real-world inference workloads and configurations.
Velocity · 7d
+1.7
★ / day
Trend
→steady
star history
GuideLLM is a platform for evaluating how language models perform under production-like workloads. It simulates end-to-end interactions with OpenAI-compatible and vLLM-native servers, generates realistic workload patterns, and measures SLO compliance. The tool helps developers identify performance bottlenecks and optimize their LLM inference deployments.