A browser extension that actually runs an LLM inside your browser

What it does

Gemma Gem is a Chrome extension that downloads and runs Google’s Gemma 4 model locally using WebGPU. It adds a chat overlay to every page you visit, and the model can inspect the DOM, take screenshots, click elements, fill forms, scroll, and execute arbitrary JavaScript — all without sending data to any server.

The interesting bit

The architecture is a three-way dance: an offscreen document hosts the model and agent loop (where the LLM decides which tools to call), a service worker routes messages and handles privileged operations like screenshots, and a content script manages the UI and DOM manipulation. This split is necessary because Chrome extension security contexts don’t let you do all three in one place. The model itself is quantized to q4f16 and served via Hugging Face’s transformers.js — so it’s ONNX weights running through WebGPU shaders in your browser tab.

Key highlights

• Two model sizes: ~500MB E2B and ~1.5GB E4B, both with 128K context window
• Native “thinking” mode toggle — the model can reason step-by-step before acting
• Fully rebindable keyboard shortcuts (default Alt+G to toggle, Escape to close)
• Per-site disable list, conversation reset, and persistent settings
• Built with WXT (Vite-based extension framework) and marked for chat rendering

Caveats

• Chrome/Edge 113+ with WebGPU and shader-f16 support required; Firefox and Safari need not apply
• Hardware estimates are exactly that — estimates, not benchmarked (4GB shared memory minimum for E2B, 6GB for E4B)
• Performance ranges from “slow” on integrated GPUs to “fast” on high-end discrete cards; the README is honest about this
• Long contexts add 10–20% memory overhead from KV cache on top of already hefty model weights

Verdict

Worth a look if you want an AI assistant that doesn’t phone home and you’re comfortable with Chrome-only, GPU-heavy browser extensions. Skip it if you need cross-browser support, run on modest hardware, or prefer your AI tools as SaaS with someone else’s uptime guarantees.