therealoliver/Deepdive-llama3-from-scratch
Educational project implementing the Llama3 transformer architecture from scratch in Python notebooks.
Not currently ranked — collecting fresh signals.
star history
This repository provides a step-by-step walkthrough of implementing Llama3 inference from scratch. It covers all core components of the transformer architecture including multi-head attention with RoPE positional encoding, SwiGLU activation, RMS normalization, KV-cache optimization, and tokenization. The project is structured as Jupyter Notebooks with detailed code annotations and dimension tracking to help learners understand each computation step.
Frequently asked
- What is therealoliver/Deepdive-llama3-from-scratch?
- Educational project implementing the Llama3 transformer architecture from scratch in Python notebooks.
- Is Deepdive-llama3-from-scratch open source?
- Yes — therealoliver/Deepdive-llama3-from-scratch is open source, released under the MIT license.
- What language is Deepdive-llama3-from-scratch written in?
- therealoliver/Deepdive-llama3-from-scratch is primarily written in Jupyter Notebook.
- How popular is Deepdive-llama3-from-scratch?
- therealoliver/Deepdive-llama3-from-scratch has 632 stars on GitHub.
- Where can I find Deepdive-llama3-from-scratch?
- therealoliver/Deepdive-llama3-from-scratch is on GitHub at https://github.com/therealoliver/Deepdive-llama3-from-scratch.