Baidu's heavy-duty training stack for LLMs and VLMs

What it does

PaddleFormers is a large-model training library built on Baidu’s PaddlePaddle framework. It implements over 100 large language and vision-language models—ranging from DeepSeek-V3 and Qwen3 to Baidu’s own ERNIE-4.5—and wraps them in a full training pipeline that spans pre-training, supervised fine-tuning, LoRA, and DPO. The project essentially acts as the Paddle ecosystem’s counterpart to Hugging Face Transformers, but with a heavier emphasis on distributed training throughput.

The interesting bit

Rather than simply porting model architectures, the library integrates deeply with PaddlePaddle’s runtime to squeeze training performance via FP8 low-precision training, communication-computation overlap, and fine-grained memory-compute balancing. It also explicitly targets domestic Chinese AI chips—Kunlun P800, Tianshu Tianqiao 150, and Muxi C550—which is unusual among globally visible training frameworks.

Key highlights

• Claims to outperform Megatron-LM on training throughput for DeepSeek-V3 and GLM-4.5-Air, per the README.
• Supports a full lifecycle from pre-training to post-training, including CPT, SFT, SFT-LoRA, DPO, and DPO-LoRA.
• Exports finished weights in Safetensors format, so models slot into vLLM, SGLang, or FastDeploy without conversion.
• Includes sequence-level optimizations like Data Packing and Padding Free to reduce idle GPU time on variable-length inputs.
• Maintains official training recipes for Baidu’s own models, including ERNIE-4.5, ERNIE-4.5-VL, and PaddleOCR-VL.

Caveats

• The README is predominantly Chinese, so non-Chinese speakers may struggle to parse installation notes and feature details.
• The “100+ models” claim is asserted but not fully enumerated in the visible table; the complete capability matrix lives in a separate document.
• Training requires CUDA 12.0+ and specific PaddlePaddle builds, suggesting a tightly coupled and potentially brittle dependency stack.

Verdict

If you are already committed to the PaddlePaddle ecosystem or required to run on domestic Chinese accelerators, this is a serious option. Teams entrenched in PyTorch, Megatron, or Hugging Face are unlikely to find enough reason to migrate.