Convolutional layers trade matrix math for B-splines

What it does This repository implements a KAN_Convolutional_Layer that replaces the standard dot-product kernel operation with learnable B-spline activations applied to each pixel in the kernel window, then summed. The authors benchmark several hybrid architectures—KAN convolutions paired with MLPs or KAN dense layers—against classic CNNs, tracking accuracy, parameter counts, and per-epoch training time across small-to-medium models.

The interesting bit The twist is literal: instead of multiplying weights and inputs, the kernel learns a separate non-linear function for every element of its receptive field. The authors note that this makes the kernel equivalent to a tiny KAN linear layer, which sounds elegant but explodes the parameter count to K²(gridsize + 2) and slows training by nearly an order of magnitude.

Key highlights

• Replaces the convolution dot product with per-pixel learnable B-spline activations (ϕ_i) and a sum.
• Parameter count grows as K²(gridsize + 2) versus K² for a standard convolution.
• On small models, KAN convolutions paired with an MLP can edge out classic CNNs; on larger ones, the advantage vanishes.
• Training time is roughly 10× slower than standard convolutions with current implementations.
• The authors admit they have not found a clear way to visualize or interpret what the learned B-splines are doing inside each kernel.

Caveats

• Accuracy gains are modest and frequently come at the cost of significantly more parameters and training times nearly an order of magnitude longer.
• The authors concede that KAN dense layers after flattening underperform plain MLPs for these image tasks, and interpretability of the learned B-splines remains unsolved.
• More complex datasets and additional architectures are explicitly listed as future work.

Verdict Worth a look if you are researching novel layer primitives or chasing the KAN trend into computer vision, but practitioners seeking a faster or leaner drop-in convolution should probably wait for more datasets and a production-ready implementation.