← all repositories

lucidrains/titans-pytorch

A PyTorch implementation of Titans, a transformer architecture with neural memory modules for long-term context retention.

2k stars Python Language ModelsML Frameworks
titans-pytorch
Velocity · 7d
+3.8
★ / day
Trend
steady
star history

This repository provides an unofficial implementation of the Titans architecture from the paper ‘Learning to Memorize at Test Time’. It implements a NeuralMemory module using MLP-based neural memory, and a MemoryAsContextTransformer that augments standard transformers with persistent and long-term memory tokens. The implementation supports test-time training adaptations and includes configurations for segment-level local attention combined with memory retrieval.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.