Chinese LLaMA variants that skipped the LoRA shortcut

What it does Linly publishes Chinese-adapted LLaMA and Falcon models, plus the ChatFlow instruction-tuned dialogue variants built on top of them. The project also trains Linly-OpenLLaMA from scratch on a 1 TB Chinese-English corpus, offering an Apache 2.0 alternative to the GPL-licensed LLaMA lineage. Everything—data prep, full training, evaluation, and format conversion scripts—is open-sourced alongside the weights.

The interesting bit While much of the community settled for parameter-efficient adapters, Linly opted for full-parameter tuning on the base models, which actually retunes the underlying representations for Chinese rather than grafting on an adapter layer. The OpenLLaMA branch is the real escape hatch: trained from scratch with a Chinese-optimized tokenizer, it sidesteps LLaMA’s viral GPL v3.0 license entirely.

Key highlights

• Full-tuned Chinese LLaMA 1/2 (7B through 70B) and Falcon-7B base models, not just LoRA adapters
• Linly-OpenLLaMA (3B/7B/13B) trained from scratch on 1TB of Chinese-English text with a hybrid character-word tokenizer
• Complete reproducibility: data preparation, pre-training, instruction fine-tuning, and evaluation code are all included
• Dual-format weights in TencentPretrain and HuggingFace formats, with conversion scripts provided
• Quantized inference support for both CUDA servers and edge devices via companion tools

Caveats

• LLaMA-derived weights are GPL v3.0 and strictly non-commercial; only the Falcon and OpenLLaMA variants carry the permissive Apache 2.0 license
• Chinese-LLaMA-2 is explicitly still iterating, with weights updated periodically, so pinned results may drift
• Deployment and quantization rely on external repositories like llama_inference, so this repo is more model zoo and training log than turnkey runtime

Verdict Grab it if you need Chinese base models with documented provenance and training code. Look elsewhere if you want a batteries-included inference server.