This repository provides code to fine-tune Large Language Models on custom datasets, specifically using LoRA, QLoRA, and DeepSpeed techniques. It supports multiple model architectures including GPT-J, Falcon, LLaMA, LLaMA2, and MPT. The repo includes Docker-based GPU workflows for managing the training environment and a quotes dataset formatted for fine-tuning. It also incorporates code from the finetune-gpt2xl project with modifications to support additional models and training methods.