mallorbc/Finetune_LLMs
A repository for fine-tuning causal LLMs with support for LoRA, QLoRA, and DeepSpeed training methods.
Not currently ranked — collecting fresh signals.
star history
This repository provides code to fine-tune Large Language Models on custom datasets, specifically using LoRA, QLoRA, and DeepSpeed techniques. It supports multiple model architectures including GPT-J, Falcon, LLaMA, LLaMA2, and MPT. The repo includes Docker-based GPU workflows for managing the training environment and a quotes dataset formatted for fine-tuning. It also incorporates code from the finetune-gpt2xl project with modifications to support additional models and training methods.
Frequently asked
- What is mallorbc/Finetune_LLMs?
- A repository for fine-tuning causal LLMs with support for LoRA, QLoRA, and DeepSpeed training methods.
- Is Finetune_LLMs open source?
- Yes — mallorbc/Finetune_LLMs is open source, released under the AGPL-3.0 license.
- What language is Finetune_LLMs written in?
- mallorbc/Finetune_LLMs is primarily written in Python.
- How popular is Finetune_LLMs?
- mallorbc/Finetune_LLMs has 465 stars on GitHub.
- Where can I find Finetune_LLMs?
- mallorbc/Finetune_LLMs is on GitHub at https://github.com/mallorbc/Finetune_LLMs.