skyzh/tiny-llm
A hands-on course teaching systems engineers to build a minimal vLLM-like LLM inference engine from scratch using Python and MLX.
Not currently ranked — collecting fresh signals.
star history
This repository provides a week-long course on LLM serving infrastructure for systems engineers. It covers implementing core LLM components (attention, RoPE, QK norm) in pure Python without high-level neural network APIs, then builds a simplified vLLM-style inference system with KV caching, continuous batching, and flash attention optimizations. The course uses Qwen3 models running on Apple Silicon via MLX framework.
Frequently asked
- What is skyzh/tiny-llm?
- A hands-on course teaching systems engineers to build a minimal vLLM-like LLM inference engine from scratch using Python and MLX.
- Is tiny-llm open source?
- Yes — skyzh/tiny-llm is open source, released under the Apache-2.0 license.
- What language is tiny-llm written in?
- skyzh/tiny-llm is primarily written in Python.
- How popular is tiny-llm?
- skyzh/tiny-llm has 4.4k stars on GitHub.
- Where can I find tiny-llm?
- skyzh/tiny-llm is on GitHub at https://github.com/skyzh/tiny-llm.