A living index of who beat which NLP benchmark

What it does

NLP-progress is a community-maintained knowledge base that maps NLP tasks to their benchmark datasets and best-known results. It covers classic workhorses like dependency parsing and named entity recognition alongside newer benchmarks such as reading comprehension, with English leading the way and over a dozen other languages represented. When a public leaderboard already exists, the repo acts as a signpost rather than a mirror. The entire collection can also be extracted into structured JSON for programmatic use.

The interesting bit

This is essentially a curated wiki with editorial standards: contributions must cite published papers, and datasets need to have been evaluated by someone other than their original authors. That level of gatekeeping is rare for a crowdsourced repository. It also refreshingly admits its own blind spots, publishing a wish list of missing tasks that includes discourse parsing and knowledge base population.

Key highlights

• Catalogs 40+ English tasks and tasks across over a dozen other languages
• Requires published paper results and externally validated datasets
• Links to official and unofficial code implementations in result tables
• Exportable to machine-readable JSON with parsed tasks and SOTA tables
• Published as a static site at nlpprogress.com and nlpsota.com

Caveats

• Coverage is heavily skewed toward English; most other languages have only a few tasks
• Several areas remain explicitly unmapped, including keyphrase extraction and frame-semantic parsing
• Relies on manual pull requests to update Markdown tables, so freshness depends entirely on contributor activity

Verdict

A useful first stop for researchers who need a quick map of NLP benchmarks and baselines; less helpful if you want an automatically updating, interactive leaderboard.