lucidrains/titans-pytorch
A PyTorch implementation of Titans, a transformer architecture with neural memory modules for long-term context retention.

This repository provides an unofficial implementation of the Titans architecture from the paper ‘Learning to Memorize at Test Time’. It implements a NeuralMemory module using MLP-based neural memory, and a MemoryAsContextTransformer that augments standard transformers with persistent and long-term memory tokens. The implementation supports test-time training adaptations and includes configurations for segment-level local attention combined with memory retrieval.