kyegomez/BitNet
A PyTorch implementation of 1-bit transformer LLMs that binarizes weight matrices for efficient training and inference.
Not currently ranked — collecting fresh signals.
star history
This repository provides a PyTorch implementation of BitNet, a method for training large language models with 1-bit weights. The core approach replaces standard nn.Linear layers with custom BitLinear modules that apply layer normalization, binarization via abs-max quantization, and dequantization. The architecture demonstrates that scaling transformer models with 1-bit weights is viable, significantly reducing memory footprint and computational cost while maintaining model quality.
Frequently asked
- What is kyegomez/BitNet?
- A PyTorch implementation of 1-bit transformer LLMs that binarizes weight matrices for efficient training and inference.
- Is BitNet open source?
- Yes — kyegomez/BitNet is open source, released under the MIT license.
- What language is BitNet written in?
- kyegomez/BitNet is primarily written in Python.
- How popular is BitNet?
- kyegomez/BitNet has 1.9k stars on GitHub.
- Where can I find BitNet?
- kyegomez/BitNet is on GitHub at https://github.com/kyegomez/BitNet.