google-research/rliable
Python library for statistically rigorous evaluation of reinforcement learning and machine learning benchmarks using bootstrap confidence intervals and aggregate metrics.
Not currently ranked — collecting fresh signals.
star history
rliable provides tools for reliable evaluation on RL and ML benchmarks even with limited runs. It implements stratified bootstrap confidence intervals to quantify uncertainty in aggregate performance, and offers alternative aggregate metrics like Interquartile Mean that are more robust to outliers than simple means or medians. The library also supports performance profile visualizations showing score distributions across tasks and runs.
Frequently asked
- What is google-research/rliable?
- Python library for statistically rigorous evaluation of reinforcement learning and machine learning benchmarks using bootstrap confidence intervals and aggregate metrics.
- Is rliable open source?
- Yes — google-research/rliable is open source, released under the Apache-2.0 license.
- What language is rliable written in?
- google-research/rliable is primarily written in Jupyter Notebook.
- How popular is rliable?
- google-research/rliable has 879 stars on GitHub.
- Where can I find rliable?
- google-research/rliable is on GitHub at https://github.com/google-research/rliable.