Adamdad/kat
A PyTorch implementation of Kolmogorov-Arnold Transformer (KAT), replacing MLP layers in Vision Transformers with KAN layers for image classification.

This repository provides a PyTorch implementation of KAT, a novel transformer architecture that integrates Group-Rational Kolmogorov-Arnold Networks (GR-KANs) for large-scale vision tasks. It replaces the standard MLP layers in Vision Transformers with learnable spline-based activation functions. The implementation includes CUDA-optimized rational base functions and grouped weight sharing for efficiency. The model achieves competitive performance on ImageNet classification.