Lightning-AI/lit-llama
Independent open-source implementation of the LLaMA language model built on nanoGPT.
Not currently ranked — collecting fresh signals.
star history
This repository provides a complete implementation of LLaMA pretraining, fine-tuning, and inference code under Apache 2.0. It supports flash attention for efficient computation, Int8 and GPTQ 4-bit quantization for reduced memory footprint, and adapter-based fine-tuning methods including LoRA and LLaMA-Adapter. The implementation is built on top of nanoGPT and enables both model training from scratch and fine-tuning of existing LLaMA weights.
Frequently asked
- What is Lightning-AI/lit-llama?
- Independent open-source implementation of the LLaMA language model built on nanoGPT.
- Is lit-llama open source?
- Yes — Lightning-AI/lit-llama is open source, released under the Apache-2.0 license.
- What language is lit-llama written in?
- Lightning-AI/lit-llama is primarily written in Python.
- How popular is lit-llama?
- Lightning-AI/lit-llama has 6.1k stars on GitHub.
- Where can I find lit-llama?
- Lightning-AI/lit-llama is on GitHub at https://github.com/Lightning-AI/lit-llama.