Alpaca-taming ChatGLM-6B on single-consumer GPUs

What it does

This repo is a training and inference toolkit for Tsinghua’s ChatGLM-6B and ChatGLM2. It converts the Alpaca dataset, tokenizes instruction-response pairs, and fine-tunes the model using LoRA so only small adapter weights are updated, not the full six-billion-parameter base model. Pre-trained LoRA checkpoints are already published on Hugging Face for those who want to skip the GPU fan noise.

The interesting bit

The project is mostly practical glue code—scripts and notebooks that wire ChatGLM to the standard Alpaca and PEFT/LoRA ecosystem—but it exposes both ChatGLM v1 and v2 through a single --version flag, letting you swap base models without rewriting the pipeline. The README bills it as a “low-cost ChatGPT” alternative, though “low cost” still means a graphics card with at least 16 GB of VRAM.

Key highlights

• LoRA adapters for ChatGLM-6B and ChatGLM2, keeping the base model frozen.
• Pre-trained Alpaca and Alpaca-zh-en LoRA weights published on Hugging Face.
• Includes data-wrangling scripts and a Colab notebook for end-to-end fine-tuning.
• Supports both ChatGLM v1 and v2 via a --version flag.
• Targets single-GPU setups, though you’ll need at least 16 GB of VRAM.

Caveats

• RLHF stages (reward model and PPO) are outlined in the README but not actually implemented yet; the TODO list admits as much.
• Native Chinese dataset support is still on the TODO list; current examples focus on the English Alpaca set.
• The bitsandbytes dependency is picky—older versions hit an undefined-symbol crash, so the README pins a specific release.

Verdict

Grab this if you want a no-frills LoRA pipeline for ChatGLM-6B/2 and own a GPU with 16 GB or more. Skip it if you need a production-ready RLHF stack or are hoping to train on a laptop integrated GPU.