lucidrains/g-mlp-pytorch
PyTorch implementation of gMLP, an all-MLP architecture that replaces attention mechanisms in Transformers.

Velocity · 7d
+0.2
★ / day
Trend
→steady
star history
This repository provides a PyTorch implementation of gMLP, a deep learning model architecture from a Google research paper that replaces attention mechanisms with spatial gating in MLPs. It includes classes for masked language modeling and vision classification tasks, supporting both standard gMLP and an attention-augmented variant (aMLP).