Skip the API Bill: A Local-Only AI Stack for Document Q&A

What it does This is boilerplate that wires up a local document Q&A pipeline inside a Next.js app. It generates embeddings in-process with Transformer.js, stores them in a local Supabase pgvector database, and answers questions using Ollama for inference. The goal is a working AI feature that never leaves your machine.

The interesting bit The unusual angle is the fully local loop: even the embeddings happen via Transformer.js and all-MiniLM-L6-v2 without an API call, and the vector store runs through the Supabase CLI on your laptop. That keeps the stack entirely offline and the operating cost at zero.

Key highlights

• LLM inference runs locally via Ollama.
• Embeddings are generated in-process using Transformer.js and all-MiniLM-L6-v2.
• Vector storage and retrieval use a local Supabase pgvector instance.
• LangChain.js orchestrates the retrieval and Q&A chain.
• The project can be extended to cloud providers like OpenAI or Pinecone by following the upstream a16z starter kit patterns.

Caveats

• The README is heavy on quickstart steps and light on architectural detail.
• Scaling characteristics and performance limits for the local Supabase + Ollama combo are not discussed.
• Cloud migration paths rely on documentation from the separate a16z AI Starter Kit.

Verdict A solid starting point for developers who want to prototype document Q&A without API keys or cloud bills, provided you do not mind running a database on your own hardware. If you need a fully managed, scalable solution out of the box, look elsewhere.