RecurrentGPT adapts LSTM-style recurrence to large language models by replacing hidden states and cell states with natural language paragraphs. It maintains long-term memory (stored summaries of generated paragraphs accessible via semantic search) and short-term memory (key information from recent timesteps). At each timestep, it combines these memory inputs in a prompt to generate the next paragraph, a plan for subsequent content, and updates both memory systems accordingly.