rasbt/LLM-workshop-2024
An educational workshop teaching how to implement and pretrain large language models from the ground up in PyTorch.
Not currently ranked — collecting fresh signals.
star history
This repository provides a structured coding workshop for understanding how LLMs work under the hood. Participants code a small GPT-like model from scratch, including the data pipeline, core transformer architecture components, and pretraining loop. The workshop also covers loading pretrained weights and finetuning LLMs using the LitGPT library. The material is based on the book Build a Large Language Model From Scratch.
Frequently asked
- What is rasbt/LLM-workshop-2024?
- An educational workshop teaching how to implement and pretrain large language models from the ground up in PyTorch.
- Is LLM-workshop-2024 open source?
- Yes — rasbt/LLM-workshop-2024 is open source, released under the Apache-2.0 license.
- What language is LLM-workshop-2024 written in?
- rasbt/LLM-workshop-2024 is primarily written in Jupyter Notebook.
- How popular is LLM-workshop-2024?
- rasbt/LLM-workshop-2024 has 1.1k stars on GitHub.
- Where can I find LLM-workshop-2024?
- rasbt/LLM-workshop-2024 is on GitHub at https://github.com/rasbt/LLM-workshop-2024.