This repository provides a PyTorch implementation of BitNet, a method for training large language models with 1-bit weights. The core approach replaces standard nn.Linear layers with custom BitLinear modules that apply layer normalization, binarization via abs-max quantization, and dequantization. The architecture demonstrates that scaling transformer models with 1-bit weights is viable, significantly reducing memory footprint and computational cost while maintaining model quality.