Chinese financial LLaMAs fine-tuned on scraped Q&A data

What it does

Cornucopia-LLaMA-Fin-Chinese releases Apache-2.0-licensed LoRA adapters that instruction-tune LLaMA for Chinese financial question-answering. The project trains on a mix of public and crawled Chinese financial Q&A covering insurance, wealth management, stocks, and credit cards, with two released 7B variants based on Meta’s LLaMA and the Chinese-aware Linly-LLaMA. It also bundles shell scripts and data formats for further fine-tuning on custom financial corpora.

The interesting bit

The authors are refreshingly honest about their data quality—current outputs still hallucinate fund names mid-sentence and conflate risk profiles, but they’re already iterating with GPT-3.5/4-generated datasets and multi-task prompts to clean it up. It’s a live look at the gap between “we fine-tuned a model” and “we trust the model.”

Key highlights

• Commercially usable under Apache 2.0; ships LoRA weights, not full model redistributions.
• Two released 7B adapters: one atop standard LLaMA, one atop Linly-AI/Chinese-LLaMA-7B for better Chinese comprehension.
• Training is single-GPU friendly: the authors report 24 GB VRAM (RTX 3090/4090) is viable with reduced batch size.
• Covers vertical financial topics—insurance, funds, loans, social security—that general chatbots often fumble.
• Roadmap includes full pre-training, multi-task SFT, RLHF, and quantization, though only instruction-tuned models are available now.

Caveats

• The comparison table in the README shows outputs that drift into irrelevant fund jargon and factual contradictions, so the current release is more “promising demo” than production oracle.
• The “lightweight training framework” branding is aspirational: right now it’s standard Hugging Face fine-tuning scripts plus data templates.
• RLHF and quantization pipelines are listed in the description but marked as “coming soon” in the README.

Verdict

Worth a look if you need a commercially safe starting point for Chinese financial NLP and can tolerate some post-training guardrails. Skip it if you need battle-tested accuracy out of the box or a fully integrated RLHF toolchain today.