ray-project/llmperf
A benchmarking library for measuring LLM API performance under load.
Not currently ranked — collecting fresh signals.
star history
LLMPerf validates and benchmarks LLM APIs through load tests and correctness tests. It measures inter-token latency, generation throughput, and concurrent request handling across APIs including OpenAI-compatible endpoints and Anthropic. The library uses standardized token counting with LlamaTokenizer for consistent cross-API comparison.
Frequently asked
- What is ray-project/llmperf?
- A benchmarking library for measuring LLM API performance under load.
- Is llmperf open source?
- Yes — ray-project/llmperf is open source, released under the Apache-2.0 license.
- What language is llmperf written in?
- ray-project/llmperf is primarily written in Python.
- How popular is llmperf?
- ray-project/llmperf has 1.1k stars on GitHub.
- Where can I find llmperf?
- ray-project/llmperf is on GitHub at https://github.com/ray-project/llmperf.