Self-hosted AI that doesn't phone home

What it does

The Self-hosted AI Starter Kit is a Docker Compose template that bundles the n8n low-code platform with Ollama for local LLMs, Qdrant as a vector store, and PostgreSQL for data storage. It gives you a pre-wired local environment for building AI workflows—appointment schedulers, PDF summarizers, Slack bots—without sending documents to external APIs. Everything runs on your own hardware, with profiles for Nvidia GPUs, AMD GPUs on Linux, or plain CPU.

The interesting bit

The project is essentially curated glue: it networks existing tools together rather than inventing a new framework, and the README refreshingly admits it is not optimized for production. That honesty makes it a solid jumping-off point for proofs-of-concept instead of a bloated platform pretending to be one.

Key highlights

• Runs fully offline with local LLMs via Ollama and local vector search via Qdrant
• Includes pre-configured networking and storage, plus a shared folder mounted at /data/shared for local file access
• Supports GPU acceleration for Nvidia and AMD (Linux), with a CPU fallback profile
• Bundles n8n’s 400+ integrations and built-in AI nodes like AI Agent and Text Classifier
• Ships with a pre-loaded chat workflow to verify the stack is alive

Caveats

• Apple Silicon users cannot pass the GPU into Docker, so they must either run Ollama natively on macOS and bridge to the container, or accept CPU inference speeds
• The maintainers note the kit is designed for proof-of-concept work and is “not fully optimized for production environments”

Verdict

Grab this if you want to prototype self-hosted AI automations without wrestling with container networking. Skip it if you need a battle-hardened production platform or turnkey multi-node orchestration.