karpathy/makemore
An educational autoregressive language model that trains character-level neural networks from bigram to Transformer architectures.
Not currently ranked — collecting fresh signals.
star history
The project implements multiple neural network architectures for character-level language modeling, following seminal papers including the Transformer architecture from Vaswani et al. 2017. It trains on text data to generate new examples similar to the training set, supporting bigrams, MLP, RNN, LSTM, GRU, and Transformer models. Built with PyTorch as the sole dependency and designed primarily for educational demonstration purposes.
Frequently asked
- What is karpathy/makemore?
- An educational autoregressive language model that trains character-level neural networks from bigram to Transformer architectures.
- Is makemore open source?
- Yes — karpathy/makemore is open source, released under the MIT license.
- What language is makemore written in?
- karpathy/makemore is primarily written in Python.
- How popular is makemore?
- karpathy/makemore has 4.1k stars on GitHub.
- Where can I find makemore?
- karpathy/makemore is on GitHub at https://github.com/karpathy/makemore.