Your browser is the entire AI backend

What it does

Secret Llama is a React chat interface that downloads open-source models—such as Llama 3 8B and Mistral 7B—straight into your browser. It leans on the WebLLM engine to handle inference via WebGPU, so your prompts never leave the machine. Once a model is cached, it even works offline, making it a genuinely local alternative to hosted chatbots.

The interesting bit

This is largely a polished UI wrapper around WebLLM, but that glue is the point: it abstracts away model fetching and GPU acceleration into a familiar chat window. Here, “serverless” means there is literally no server, not just a cloud provider you don’t see.

Key highlights

• Supports models ranging from a 600MB TinyLlama to a 4.3GB Llama 3 8B with zero backend.
• Requires only a modern WebGPU browser; no install or account needed.
• Runs fully offline once models are cached.
• Explicitly credits WebLLM for the heavy lifting under the hood.

Caveats

• WebGPU support is still patchy: Chrome and Edge work today, while Firefox and Safari need experimental flags flipped.
• Heavier models need serious RAM—up to 4.3GB for Llama 3—so low-end hardware is stuck with smaller options.
• The author flags initial model loading as slow and lists speeding it up as a top contribution target.

Verdict

Try it if you want a no-friction way to test local LLMs without touching a terminal. Give it a pass if you’re on an underpowered machine or a browser that treats WebGPU as a science experiment.