ray-project/llmperf
A benchmarking library for measuring LLM API performance under load.
Velocity · 7d
+1.2
★ / day
Trend
→steady
star history
LLMPerf validates and benchmarks LLM APIs through load tests and correctness tests. It measures inter-token latency, generation throughput, and concurrent request handling across APIs including OpenAI-compatible endpoints and Anthropic. The library uses standardized token counting with LlamaTokenizer for consistent cross-API comparison.