Fudan’s legal LLM reasons in syllogisms and cites its sources

What it does

DISC-LawLLM is a family of Chinese legal language models—13B and 7B parameters—fine-tuned for legal consultation, document drafting, case prediction, and exam prep. It pairs supervised fine-tuning with a retrieval module backed by roughly 800 national and local statutes and 24,000 exam questions. The project also ships a 403K-sample training dataset and an evaluation benchmark so other researchers can replicate or extend the work.

The interesting bit

The team structured part of the instruction data around legal syllogism—the classical theory of legal reasoning—rather than generic Q&A formats, making a rare explicit attempt to teach an LLM judicial logic instead of mere pattern matching. They report Lawbench scores that place the model second only to GPT-4 and ahead of GPT-3.5, though the exact methodology lives in their technical report.

Key highlights

• Two model sizes: a 13B build on Baichuan-13B-Base and a newer 7B build on Qwen2.5-Instruct, the latter meant to fix compatibility headaches with modern transformers and vLLM.
• A retrieval-augmented generation layer built atop Langchain-Chatchat, querying a knowledge base of 800+ laws and 24,000 legal exam items.
• A 403K-sample supervised-fine-tuning dataset (DISC-Law-SFT) split into “Pair” samples for legal reasoning and “Triplet” samples for external-knowledge tasks.
• An open evaluation benchmark, DISC-Law-Eval, plus reported Lawbench results that the authors say rank above GPT-3.5 and every other tested legal model.
• Academic backing: the underlying work was accepted as a long paper at DASFAA 2024 (CCF-B).

Caveats

• The original 13B model is pinned to an old transformers release (4.29.1) because newer versions dropped support for the Baichuan architecture; the 7B Qwen-based release is the pragmatic escape hatch.
• The retrieval module is essentially a Langchain-Chatchat integration with a curated Chinese legal corpus—useful, but not a novel retrieval architecture.
• Most documentation and dataset metadata are in Chinese, so non-Chinese speakers will need translation help to navigate the training data or fine-tuning guides.

Verdict

Worth a look if you are building Chinese legal-tech tools or researching domain-specific reasoning datasets; skip it if you need a general-purpose multilingual assistant or a fully novel retrieval architecture.