← all repositories

SHI-Labs/NATTEN

Open-source library providing optimized CUDA kernels for multi-dimensional sparse self-attention in vision transformers.

Velocity · 7d
+0.6
★ / day
Trend
steady
collecting data…
star history

NATTEN implements Neighborhood Attention and related sparse attention variants as fused CUDA kernels compatible with PyTorch. It targets NVIDIA GPUs across multiple architectures and provides both training and inference kernels. The library enables efficient sliding-window self-attention that introduces locality and sparsity similar to convolutions, primarily supporting 2D and 3D feature map layouts used in vision transformer architectures.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.