From Pretraining to GRPO: A One-Stop Shop for Domain LLMs

What it does

This project replicates the full ChatGPT-style model development cycle for domain-specific large language models, using medical AI as its reference domain. It orchestrates continued pretraining on domain documents, supervised fine-tuning with instruction data, and a full menu of alignment methods—RLHF, DPO, ORPO, GRPO, and standalone on-policy distillation (OPD)—to produce specialized models like its released Ziya-LLaMA and Baichuan medical variants. It also ships with supporting utilities: a ChatPDF-style RAG demo, role-play dialogue generation scripts, and tool-calling fine-tuning formats.

The interesting bit

Rather than treating alignment as a single technique, the repo collects essentially every major post-training method from the last two years into one pipeline, letting you swap between RLHF, DPO, ORPO, or GRPO like interchangeable lenses. It also keeps pace with new architectures—Qwen3.5 MoE, LLaMA 3, Mixtral—suggesting the “Medical” branding is increasingly a demo case for what is really a general domain-adaptation engine.

Key highlights

• Full four-stage pipeline: continued pretraining (PT), supervised fine-tuning (SFT), reward modeling and reinforcement learning (RLHF), plus DPO, ORPO, GRPO, and OPD distillation
• Broad model support across LLaMA 2/3, Qwen 2/2.5/3.5 (including MoE variants), Baichuan, ChatGLM, Bloom, and Mixtral 8x7B
• Published model weights and LoRA adapters on Hugging Face, trained on a 2.4M-entry Chinese-English medical dataset and ShareGPT dialogue data
• Extras: FlashAttention-2 and LongLoRA context extension, NEFTune embedding noise, role-play data generation via multiple LLM providers, and tool-calling/function-call fine-tuning
• Apache 2.0 license

Caveats

• The README claims medical QA “效果有提升” (effect improved) but offers no quantitative benchmarks or clinical safety evaluations in the provided sources
• Some advanced features are explicitly experimental: the OPD distillation module, for instance, is noted as a first release that does not yet jointly train with PPO/GRPO
• Documentation and release notes are predominantly in Chinese, with English available but seemingly secondary

Verdict

Worth a look if you are building a Chinese or vertical-domain LLM and want a ready-made training curriculum instead of assembling Hugging Face scripts by hand. Skip it if you need a validated, turnkey medical diagnostic API or rigorous clinical-grade model evaluation.