GAIR-NLP/MathPile
A 9.5B-token math-centric pre-training corpus for training LLMs on mathematical reasoning.
Not currently ranked — collecting fresh signals.
star history
MathPile is a large-scale pre-training corpus designed for training language models on mathematical content. Released at NeurIPS D&B 2024, it contains approximately 9.5 billion tokens of high-quality, diverse math-centric data from multiple sources including web math, textbooks, and arXiv papers. The repository includes data processing scripts used to construct and clean the corpus, enabling researchers to reproduce the dataset or apply similar pipelines to other domains.
Frequently asked
- What is GAIR-NLP/MathPile?
- A 9.5B-token math-centric pre-training corpus for training LLMs on mathematical reasoning.
- Is MathPile open source?
- Yes — GAIR-NLP/MathPile is open source, released under the Apache-2.0 license.
- What language is MathPile written in?
- GAIR-NLP/MathPile is primarily written in Python.
- How popular is MathPile?
- GAIR-NLP/MathPile has 419 stars on GitHub.
- Where can I find MathPile?
- GAIR-NLP/MathPile is on GitHub at https://github.com/GAIR-NLP/MathPile.