← all repositories
xmu-xiaoma666/External-Attention-pytorch

A Lego box for attention mechanisms: 37 ways to look at pixels

Standalone PyTorch implementations of every attention block you've seen in papers, stripped of task-specific scaffolding so you can actually read the code.

12.2k stars Python ML FrameworksOther AI
External-Attention-pytorch
Velocity · 7d
+6.6
★ / day
Trend
steady
star history

What it does

This repo extracts the core building blocks from computer-vision papers—attention modules, MLP layers, re-parameterization tricks, convolutions—and re-implements each as a clean, self-contained PyTorch class. No full training pipelines, no detection-framework glue. Just the Lego brick.

The interesting bit

The author frames it as solving the “ten lines of core code buried in ten thousand lines of repo” problem. Each module is importable via pip install fightingcv-attention and runnable in isolation, which makes it useful for ablation studies or for staring at an attention map until it finally clicks.

Key highlights

  • 37+ attention mechanisms, from classic SE and CBAM to newer entries like CoAtNet, MobileViTv2, and UFO attention
  • Also covers backbones (ResNet, ConvNeXtV2, EfficientFormer), MLP blocks (MLP-Mixer, gMLP, RepMLP), re-parameterization (RepVGG, ACNet), and conv variants (Involution, DynamicConv)
  • Each entry links to the arXiv paper and often to a Chinese blog walkthrough
  • Installable as fightingcv-attention or cloned for direct model.* imports

Caveats

  • No training scripts, benchmarks, or pre-trained weights provided—this is strictly reference code
  • README is mostly in Chinese; English version exists but may lag
  • Code quality and docstring coverage likely vary across 70+ contributed modules

Verdict

Grab this if you’re a researcher prototyping architectures or a student who learns by tweaking. Skip it if you need drop-in, battle-tested layers with unit tests and performance guarantees.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.