This repository categorizes academic papers on knowledge distillation from 2014 to 2021, covering various forms of knowledge transfer including logits-based, intermediate layer-based, graph-based, and structural knowledge approaches. It also includes applications of knowledge distillation combined with other techniques such as GANs, meta-learning, reinforcement learning, and AutoML. The collection serves as a reference for model compression and efficient training methodologies in deep learning.