Rust LLM pipelines: from document chunks to tool-wielding agents

What it does Swiftide is a Rust library for building LLM applications. It offers streaming pipelines that ingest documents—chunking code with tree-sitter, generating metadata, embedding—and drop them into vector stores like Qdrant or LanceDB. On the other side, it can query those stores with hybrid search and subquestion generation, or run agents that call tools via a procedural macro. The whole thing is built on async, parallel primitives so you are not waiting for one file to finish before the next starts.

The interesting bit Instead of treating retrieval and agents as separate ecosystems, Swiftide uses the same pipeline language for both. You can index a codebase, query it, then hand the same LLM client to an agent that executes shell commands—all via a #[swiftide::tool] macro and Jinja-like prompt templating. It is also part of bosun.ai, an autonomous code-improvement platform, which means the abstractions are being stress-tested on actual agentic workloads.

Key highlights

• Streaming indexing with async, parallel processing; loaders cover files, Kafka, Parquet, and Fluvio streams.
• Experimental query pipelines supporting similarity and hybrid search, plus RAGAS evaluation.
• An experimental agent framework with a #[swiftide::tool] macro for defining LLM-callable functions.
• Integrations with OpenAI, Anthropic, Gemini, Groq, Ollama, Qdrant, Redis, LanceDB, Postgres, and tree-sitter.
• Built-in tracing and Langfuse export for observability.

Caveats

• The query pipeline and agent framework are explicitly marked experimental.
• The project is under heavy development with breaking changes, and the README warns that documentation may lag behind features.

Verdict Worth a look if you are building Rust-native LLM applications and want a structured, pipeline-driven alternative to piecing together raw SDKs. If you need stable, battle-tested abstractions today—or you are not committed to Rust—this is probably not your stop yet.