Fine-tuning ChatGLM to cite Chinese law

What it does

LawGPT_zh (named 獬豸) is a domain-specific chatbot built by fine-tuning ChatGLM-6B with LoRA on Chinese legal text. It answers everyday legal questions—credit-card debt, inheritance disputes, workplace harassment—and offers both direct generation and a retrieval-augmented mode that searches legal text embeddings before responding.

The interesting bit

Instead of scraping random forums, the authors use a “Reliable-Self-Instruction” pipeline: they feed ChatGPT specific legal articles from a core Chinese statute book, force it to generate questions anchored to those texts, and then answer them. This creates synthetic training data structurally tied to real law, and they have open-sourced the resulting 92k scenario pairs complete with statutory references.

Key highlights

• Built on ChatGLM-6B with 16-bit LoRA; runs on a single GPU with 15 GB+ VRAM
• Two inference modes: direct generation and retrieval-augmented generation (demo_r.py)
• Training corpus includes 92k scenario Q&A pairs explicitly citing Chinese legal statutes, plus 52k single-round dialogues cleaned from real lawyer-client data
• Version 1.1-alpha adds statutory citations to answers, aiming for reliability over generic platitudes
• Uses “Reliable-Self-Instruction” to constrain synthetic data generation to specific legal texts

Caveats

• Still alpha software; the knowledge-QA dataset is marked “coming soon” and the README itself is truncated mid-example
• Model weights and datasets are distributed via Baidu Pan links rather than standard model hubs
• Even the showcased retrieval example contains a garbled statute citation (“第一千一百六一号”), suggesting citation accuracy remains uneven

Verdict

A useful starting point for researchers or developers building Chinese legal NLP tools who need a domain-adapted ChatGLM baseline and structured training data. Anyone seeking actual legal counsel should look elsewhere—this is a research prototype, not a substitute for a qualified lawyer.