SHI-Labs/NATTEN
Open-source library providing optimized CUDA kernels for multi-dimensional sparse self-attention in vision transformers.
Velocity · 7d
+0.6
★ / day
Trend
→steady
collecting data…
star history
NATTEN implements Neighborhood Attention and related sparse attention variants as fused CUDA kernels compatible with PyTorch. It targets NVIDIA GPUs across multiple architectures and provides both training and inference kernels. The library enables efficient sliding-window self-attention that introduces locality and sparsity similar to convolutions, primarily supporting 2D and 3D feature map layouts used in vision transformer architectures.