Build a toy LLM cosmos from scratch, no black boxes allowed

What it does

Tiny-Universe is a set of Jupyter-based tutorials that walk you through hand-building the core components of a modern AI stack. It covers an LLM (TinyLlama3), a diffusion model (TinyDiffusion), a Transformer, plus surrounding infrastructure: RAG (including GraphRAG), an Agent system, and an evaluation framework (TinyEval). Everything is built from first principles using PyTorch and NumPy, deliberately avoiding the convenience of pre-built frameworks so you can see the wiring.

The interesting bit

The project treats “white box” understanding as a feature, not a side effect. It doesn’t just build a model; it builds the entire lifecycle—pre-training a Llama3-compatible model on roughly 2 GB of VRAM, then evaluating it with custom metrics (including Chinese Gaokao math problems), augmenting it with a hand-rolled RAG retriever, and wrapping it in a minimal ReAct agent. The Qwen-Blog module even dissects a real production architecture by tracing a single input tensor through GQA, RoPE, and attention masks.

Key highlights

• Full-stack scope: model pre-training, diffusion, RAG, GraphRAG, agent tooling, and evaluation in one curriculum.
• Low resource footprint: TinyLlama3 pre-training targets ~2 GB VRAM; TinyDiffusion claims a two-hour pre-training cycle.
• Code is intentionally minimal and “white box,” aimed at readability over production robustness.
• Includes video walkthroughs (Tencent Meeting recordings) for several modules.
• Active expansion: recent additions include GraphRAG and academic reproductions like CDDRS.

Caveats

• All materials and explanations are in Chinese, which limits the audience.
• The TinyAgent module is explicitly described as more of a minimal tool-calling demo than a full autonomous agent, with more advanced architectures still planned.
• Some performance claims, such as the two-hour diffusion pre-training, appear in release notes without detailed reproducibility benchmarks in the documentation.

Verdict

Ideal for Chinese-speaking developers with basic deep learning knowledge who are tired of framework magic and want to see the wiring behind LLMs, RAG, and agents. Skip it if you are looking for a production-ready framework or an English-language resource; this is a pedagogical repo, not a library.