charent/ChatLM-mini-Chinese
A 0.2B parameter Chinese dialogue language model with complete open-source training pipeline.

ChatLM-mini-Chinese is a small Chinese dialogue model trained from scratch, covering the full LLM training lifecycle: dataset sourcing and cleaning, tokenizer training with sentencepiece and HuggingFace tokenizers, Text-to-Text pretraining, SFT instruction fine-tuning, and DPO/RLHF preference optimization. It provides custom trainers supporting single and multi-GPU distributed training with checkpoint resumption, using HuggingFace frameworks (transformers, accelerate, trl, peft). The model can run inference on devices with as little as 512MB VRAM.