A private ChatGPT that actually lives inside your VPC

What it does

RAGstack packages open-source LLMs, the Qdrant vector database, and a web UI into a deployable retrieval-augmented generation stack. It targets teams who want a ChatGPT-like interface over their own documents without leaving their GCP, AWS, or Azure VPC. Local development relies on GPT4All running on consumer CPUs, while cloud deployments use GPU-backed Falcon-7B or Llama 2 7B.

The interesting bit

The project is refreshingly upfront about being integration-heavy: it containerizes off-the-shelf models, wires them to a Rust vector store, and wraps everything in Terraform scripts. That honesty saves you from reinventing the same plumbing, especially if your security team insists on air-gapped inference.

Key highlights

• Terraform-based deployment scripts for GCP (GKE), AWS (ECS), and Azure (AKS).
• Local mode runs on consumer CPUs via GPT4All; cloud modes use Falcon-7B or Llama 2 7B on GPUs.
• Uses Qdrant as the self-hostable vector database.
• Provides a simple server and UI for PDF upload and conversational retrieval.
• Llama-2-40b support is listed as under construction on the roadmap.

Caveats

• Azure AKS deployments depend on NVIDIA Tesla T4 node pools that aren’t available in all subscriptions.
• The README contains visible editing scars—duplicate and misnumbered steps in the Azure section.
• Stale branding lingers in badge and image URLs that still point to the old psychicapi organization, suggesting the docs haven’t fully caught up with the repo’s move to finic-ai.

Verdict

Worth a look if you need a VPC-native document chatbot and prefer assembling proven open-source parts over paying for a managed service. Skip it if you want turnkey SaaS or a novel research architecture.