A Python framework that treats LLM context as plumbing, not magic

What it does Haystack is a Python framework for building production LLM applications—RAG systems, agents, semantic search, multimodal pipelines—by composing modular components into explicit workflows. You control how documents are retrieved, ranked, filtered, and routed before they ever reach a model, with loops and branches for agentic logic.

The interesting bit Most LLM frameworks bury context engineering under abstraction layers. Haystack inverts this: retrieval, memory, and generation are first-class, traceable citizens in a pipeline graph. Swap OpenAI for a local Hugging Face model or AWS Bedrock without rewriting your orchestration logic—the model is just another node.

Key highlights

• Vendor-agnostic by design: OpenAI, Anthropic, Mistral, Cohere, Azure, local models, and more plug into the same component interface
• Built-in components for retrieval, indexing, tool calling, memory, and evaluation, plus hooks for custom ones
• Transparent architecture with explicit control over context flow—no black-box prompt assembly
• Companion project Hayhooks exposes pipelines as REST APIs or MCP servers with OpenAI-compatible endpoints
• 25K+ stars, used by Apple, Meta, NVIDIA, Airbus, and the European Commission in production systems

Caveats

• Anonymous telemetry is on by default; you must opt out explicitly via docs
• Enterprise support and managed platform are clearly upsold; the open-source project is the entry point
• README emphasizes “production-ready” but doesn’t quantify latency, throughput, or failure modes

Verdict Worth a look if you’re building non-trivial RAG or agent systems where debuggability and model portability matter. Skip it if you want a zero-config chatbot wrapper or are allergic to pipeline thinking.