megvii-research/mdistiller
PyTorch library implementing knowledge distillation methods for compressing CNN models, with official implementations of CVPR2022 and ICCV2023 papers.

A PyTorch library providing classical and state-of-the-art knowledge distillation algorithms on mainstream computer vision benchmarks including CIFAR-100, Tiny-ImageNet, and ImageNet. It includes official implementations of Decoupled Knowledge Distillation (DKD) and DOT (Distillation-Oriented Trainer), which propose improved training strategies for compressing larger teacher models into smaller student models in the teacher-student distillation paradigm.