Transformer squared is a novel framework that enables LLMs to adapt to unseen tasks without traditional fine-tuning. It uses a two-pass inference mechanism where a dispatch system identifies task properties, then dynamically mixes learned expert vectors to produce targeted behavior. Expert vectors are trained using reinforcement learning to selectively adjust singular components of model weight matrices, enabling efficient task-specific adaptation during inference.