srush/LLM-Training-Puzzles
An 8-puzzle interactive tutorial by Sasha Rush teaching distributed training of large language models across many GPUs.
Not currently ranked — collecting fresh signals.
star history
This repository contains a set of interactive Jupyter notebooks with challenges covering the technical aspects of training LLMs on thousands of GPUs. Topics include memory efficiency strategies and compute pipelining techniques for distributed neural network training. The puzzles serve as hands-on education for practitioners who rarely get to work with large-scale GPU clusters.
Frequently asked
- What is srush/LLM-Training-Puzzles?
- An 8-puzzle interactive tutorial by Sasha Rush teaching distributed training of large language models across many GPUs.
- Is LLM-Training-Puzzles open source?
- Yes — srush/LLM-Training-Puzzles is open source, released under the MIT license.
- What language is LLM-Training-Puzzles written in?
- srush/LLM-Training-Puzzles is primarily written in Jupyter Notebook.
- How popular is LLM-Training-Puzzles?
- srush/LLM-Training-Puzzles has 1.2k stars on GitHub.
- Where can I find LLM-Training-Puzzles?
- srush/LLM-Training-Puzzles is on GitHub at https://github.com/srush/LLM-Training-Puzzles.