keyvank/femtoGPT
A minimal GPT architecture implemented entirely from scratch in Rust with CPU and GPU training/inference support.
Not currently ranked — collecting fresh signals.
star history
femtoGPT is a from-scratch implementation of a Generative Pretrained Transformer written in pure Rust. It supports both training and inference on CPU and GPU (via OpenCL). All tensor operations and neural network layers are implemented manually without external ML frameworks, making it a learning tool for understanding how LLMs work internally. Gradient correctness is verified using gradient checking.
Frequently asked
- What is keyvank/femtoGPT?
- A minimal GPT architecture implemented entirely from scratch in Rust with CPU and GPU training/inference support.
- Is femtoGPT open source?
- Yes — keyvank/femtoGPT is open source, released under the MIT license.
- What language is femtoGPT written in?
- keyvank/femtoGPT is primarily written in Rust.
- How popular is femtoGPT?
- keyvank/femtoGPT has 936 stars on GitHub.
- Where can I find femtoGPT?
- keyvank/femtoGPT is on GitHub at https://github.com/keyvank/femtoGPT.