This repository contains the evaluation harness for the HumanEval dataset, a benchmark for assessing code-generating language models. It provides tooling to run model-generated Python code in a sandboxed environment and measure functional correctness against hand-written test cases. The dataset and methodology were introduced in the paper ‘Evaluating Large Language Models Trained on Code’ and is widely used to compare code-assistance models like Codex.