What it does

AnythingLLM is a self-hosted AI application that lets you chat with documents, build no-code agents, and manage vector databases through a web UI. It connects to local models via Ollama or LM Studio, or to cloud providers like OpenAI and Anthropic, and ingests PDFs, Word docs, and text files for retrieval-augmented generation.

The interesting bit

The project treats “local-first” as the default, not the exception. LanceDB ships built-in for vector storage, and the desktop app runs on macOS, Windows, and Linux without Docker gymnastics. The “Dynamic Model Routing” feature lets you define rules that automatically switch between cheap local models and premium APIs based on the conversation.

Key highlights

• Supports 30+ LLM providers and 8 vector databases, switchable per workspace
• Built-in no-code agent builder with MCP compatibility and scheduled cron jobs
• Multi-user support with permissioning (Docker deployment)
• Embeddable chat widget and browser extension as separate submodules
• Full REST API for custom integrations

Caveats

• Multi-user features and the embeddable widget are Docker-only; the desktop app is single-user
• “Hyper-configurable” means you’ll be clicking through a lot of settings panels
• The README claims “zero setup friction” but the feature matrix suggests otherwise for advanced use

Verdict

Worth a look if you want a private alternative to ChatGPT with document chat and basic agent workflows. Skip it if you need deep custom orchestration—this is a configurable GUI, not a framework.