szagoruyko/attention-transfer
PyTorch implementation of attention transfer knowledge distillation for improving convolutional neural networks.
Not currently ranked — collecting fresh signals.
star history
This repository contains PyTorch code for the ICLR 2017 paper ‘Paying More Attention to Attention’, which transfers attention maps from teacher networks to student networks to improve CNN performance. It includes activation-based attention transfer implementations for CIFAR-10 and ImageNet experiments (ResNet-18 to ResNet-34 student-teacher pairs), along with a Jupyter notebook for visualizing attention maps.
Frequently asked
- What is szagoruyko/attention-transfer?
- PyTorch implementation of attention transfer knowledge distillation for improving convolutional neural networks.
- Is attention-transfer open source?
- Yes — szagoruyko/attention-transfer is an open-source project tracked on heatdrop.
- What language is attention-transfer written in?
- szagoruyko/attention-transfer is primarily written in Jupyter Notebook.
- How popular is attention-transfer?
- szagoruyko/attention-transfer has 1.5k stars on GitHub.
- Where can I find attention-transfer?
- szagoruyko/attention-transfer is on GitHub at https://github.com/szagoruyko/attention-transfer.