Modular long-term memory for LLMs that stays light

What it does LightMem is a memory management framework for large language models and AI agents. It handles storage, retrieval, and updates of long-term memory through a modular pipeline that includes pre-compression, topic segmentation, and embedding-based retrieval. The framework supports both cloud APIs like OpenAI and DeepSeek and local inference via Ollama, vLLM, and Hugging Face Transformers.

The interesting bit Rather than treating memory as a black-box database, LightMem breaks the pipeline into swappable components—swap in llmlingua-2 or entropy-based compression, pick FAISS, Qdrant, or BM25 for retrieval, and plug in different embedders or LLM backends. That modularity is paired with a built-in evaluation harness that benchmarks its own performance against Mem0, A-MEM, and LangMem on standard datasets like LoCoMo and LongMemEval.

Key highlights

• Modular architecture with pluggable compressors, segmenters, retrievers, and LLM backends.
• Supports both cloud APIs (OpenAI, DeepSeek) and local models (Ollama, vLLM, Transformers).
• Ships with reproduction scripts and a baseline evaluation framework for LoCoMo and LongMemEval.
• Includes an MCP server for tool integration and Jupyter tutorial notebooks for travel and code-assistant scenarios.
• Accepted at ICLR 2026; actively maintained with a public roadmap including KV-cache pre-computation and multi-modal memory.

Caveats

• Several headline features remain on the todo list, including offline KV-cache pre-computation and multi-modal memory support.
• The PyPI package is listed as “coming soon,” so installation is currently source-only.

Verdict Developers building agentic applications who want a memory layer more configurable than a simple vector store should take a look. If you need a fully managed, production-grade memory service with zero setup, this is still a research framework with rough edges.