A ChatGPT Clone That Lives Entirely in Your Browser Tab

What it does

Chatty is a Next.js chat interface that pulls down open-source models—Gemma, Llama 2/3, Mistral—and executes them inside the browser tab through WebGPU. It handles markdown rendering, code highlighting, voice input, and light/dark mode, all without sending prompts to a remote server. You can also load local files (PDFs, text, or code) and query them through an in-browser embedding pipeline built on Xenova transformers and LangChain’s memory vector store.

The interesting bit

The project is mostly a polished UI layer atop the WebLLM engine. What makes it notable is the full local RAG pipeline: it embeds and queries your documents inside the browser without ever calling a server. The trade-off is a steep hardware appetite—about 3–6 GB of GPU memory just to open a chat tab.

Key highlights

• Runs 3B and 7B parameter models entirely client-side after an initial download, with no ongoing network needed.
• Performs local file embeddings and Q&A over documents using Xenova’s all-MiniLM model and an in-memory vector store.
• Mimics the UX patterns of ChatGPT and Gemini—regenerate responses, custom memory/system instructions, chat history, and voice input.
• Exports conversations to JSON or Markdown for portability.
• Requires a WebGPU-enabled browser (Chrome or Edge by default; Firefox needs manual flags).

Caveats

• The Dockerfile is explicitly noted as not yet optimized for production.
• File chat is currently limited to one document per session; multiple file embeddings are on the roadmap.
• A 7B model needs roughly 6 GB of GPU memory and smaller models struggle with file embeddings, so hardware requirements are non-trivial.

Verdict

Worth exploring if you want a private, browser-based alternative to cloud chatbots and have the GPU memory to spare; skip it if you are looking for a lightweight, production-ready server deployment.