deepseek-ai/DualPipe
A bidirectional pipeline parallelism algorithm that overlaps forward and backward computation with communication during LLM training.

Velocity · 7d
+6.4
★ / day
Trend
→steady
star history
DualPipe implements an innovative scheduling strategy for distributed training of large language models, enabling full overlap of computation and communication phases while reducing pipeline bubbles. It provides both a full bidirectional schedule and a more memory-efficient V-shape variant (DualPipeV). The algorithm is designed specifically for training DeepSeek V3 and R1 models across multiple devices.