rasbt/LLM-workshop-2024
An educational workshop teaching how to implement and pretrain large language models from the ground up in PyTorch.

Velocity · 7d
+1.5
★ / day
Trend
→steady
star history
This repository provides a structured coding workshop for understanding how LLMs work under the hood. Participants code a small GPT-like model from scratch, including the data pipeline, core transformer architecture components, and pretraining loop. The workshop also covers loading pretrained weights and finetuning LLMs using the LitGPT library. The material is based on the book Build a Large Language Model From Scratch.