Teaching PyTorch convolutions to use functions instead of weights

What it does

torch-conv-kan provides PyTorch implementations of convolutional layers where the kernel is a set of univariate learnable functions—such as B-splines, Chebyshev polynomials, wavelets, and Gram polynomials—rather than a standard matrix of weights. It bundles these layers into familiar architectures including ResNet-like (ResKANet), DenseNet-like (DenseKANet), VGG-like (VGGKAN), and U-Net-like (UKANet) models, and provides training scripts for MNIST, CIFAR, Tiny ImageNet, and ImageNet-1k. The project also releases pretrained weights for some VGG-style models on ImageNet-1k, with the best current checkpoint hitting 68.5% top-1 accuracy using bottleneck Gram polynomial convolutions.

The interesting bit

The repository treats the KAN theorem as a literal drop-in replacement for convolutions: instead of sliding a weight matrix over pixels, it slides learnable non-linear functions and sums the outputs. To keep parameter counts sane, it introduces bottleneck variants that squeeze channels with 1×1 convolutions before applying the expensive functional transformations—an idea borrowed from ResNet, but applied to polynomial splines.

Key highlights

• Broad functional menu: supports B-splines, Legendre, Chebyshev, Gram, Jacobi, and Bernstein polynomials, plus wavelets and ReLU-based KANs.
• Ready-made vision architectures: ResKANet, DenseKANet, VGGKAN, UKANet, U2KANet, and RDNet-like variants.
• Published paper and pretrained checkpoints: includes an arXiv paper and ImageNet-1k pretrained weights for several VGG KAGN models.
• Bottleneck convolutional KAN layers to reduce parameters when channel counts grow.
• Training infrastructure built on accelerate and PyTorch with scripts for CIFAR-10/100, Tiny ImageNet, and ImageNet-1k.

Caveats

• The project is explicitly marked “Under Development,” and several benchmark numbers are labeled as preliminary results with more experiments ongoing.
• The LBFGS optimizer support is noted as not well-tested.
• The author mentions limited computational resources, so additional pretrained checkpoints and larger-scale experiments are arriving gradually.

Verdict

Worth a look if you are researching alternative inductive biases for vision or want a working PyTorch testbed for convolutional KANs. Skip it if you need production-grade, battle-tested layers; this is an active research sandbox, not a finished library.