A factory line for fine-tuning 100+ open models

What it does

LLaMA Factory is an ACL 2024 project that provides a unified framework for fine-tuning over 100 large language and vision-language models. It wraps pre-training, supervised fine-tuning, reward modeling, RLHF variants such as PPO, DPO, KTO, and ORPO, and inference into one toolchain. You drive it through a zero-code CLI or a Gradio web board, and it handles quantization—LoRA, QLoRA from 2-to-8-bit, and optimizers like GaLore, BAdam, and Muon—without forcing you to wire up the kernels yourself.

The interesting bit

The project tracks new model releases with almost alarming speed, offering Day 0 support for Qwen3 and Gemma 3 and Day 1 support for Llama 4. That velocity turns it into a living compatibility layer rather than a static library, which likely explains why Amazon, NVIDIA, and Aliyun use it.

Key highlights

• Supports 100+ models including LLaMA, DeepSeek, Qwen, Mistral, and multimodal variants like LLaVA and Qwen2.5-VL.
• Quantization buffet: 2/3/4/5/6/8-bit QLoRA via AQLM, AWQ, GPTQ, and others, alongside full 16-bit tuning.
• Built-in inference through vLLM or SGLang workers, exportable to OpenAI-style APIs or Ollama modelfiles.
• Covers multimodal tasks: image understanding, visual grounding, video recognition, and audio understanding.
• Experiment tracking via LlamaBoard, TensorBoard, Weights & Biases, MLflow, and SwanLab.

Caveats

• The official documentation is explicitly marked as a work in progress.

Verdict

If you need to fine-tune or deploy open models without maintaining a private fork of the latest training stack, this is your toolkit. If you prefer writing custom training loops from scratch, it will feel like overkill.