This project implements a language model architecture capable of utilizing long-term memory. It augments a frozen pre-trained LLM backbone with a trainable side network and joint attention mechanism to retrieve and integrate relevant past context from a memory bank. The system uses Faiss-GPU for efficient vector similarity search over encoded memory chunks, enabling the model to attend to information from extended context windows.