szagoruyko/attention-transfer
PyTorch implementation of attention transfer knowledge distillation for improving convolutional neural networks.

Velocity · 7d
+0.4
★ / day
Trend
→steady
star history
This repository contains PyTorch code for the ICLR 2017 paper ‘Paying More Attention to Attention’, which transfers attention maps from teacher networks to student networks to improve CNN performance. It includes activation-based attention transfer implementations for CIFAR-10 and ImageNet experiments (ResNet-18 to ResNet-34 student-teacher pairs), along with a Jupyter notebook for visualizing attention maps.