Run LLMs, Whisper, and Kokoro without a backend

What it does

BrowserAI is a TypeScript SDK that runs LLMs, speech recognition, text-to-speech, and audio source separation directly in the browser via WebGPU. It wraps several underlying engines—MLC, Transformers.js, Flare WASM, and Demucs—behind a single API, letting you load quantized models like Llama, DeepSeek, Qwen, Whisper, and Kokoro without a server roundtrip. The library also supports structured JSON output, Web Workers for non-blocking inference, and client-side storage for embeddings and conversations.

The interesting bit

This is largely a carefully curated abstraction layer: it takes existing browser inference projects and unifies them under one BrowserAI class with pre-configured model definitions. The practical payoff is that developers get offline-capable, zero-infrastructure AI without wrestling with engine-specific APIs or quantization flags—including audio stem separation via Demucs, which is an unusual trick for a browser tab.

Key highlights

• Supports four inference backends: MLC, Transformers.js, Flare (GGUF via WASM), and Demucs
• Runs entirely client-side with WebGPU acceleration; works offline after the initial model download
• Ships with pre-configured models for text generation (Llama, DeepSeek, Qwen), speech (Whisper), TTS (Kokoro), and audio separation
• Offers structured JSON output via schemas, Web Worker support, and built-in client-side database storage
• Requires a modern WebGPU-capable browser (Chrome/Edge 113+); some models need hardware support for 16-bit float operations

Caveats

• Requires a modern WebGPU-capable browser (Chrome 113+, Edge 113+, or equivalent), and some models additionally need hardware support for shader-f16 16-bit floating-point operations
• The roadmap lists RAG, observability dashboards, and multi-model orchestration as future work, suggesting the current feature set is still expanding

Verdict

A solid bet for privacy-first web apps and no-code builders that need client-side AI without backend bills. Look elsewhere if you need broad browser compatibility or mature RAG and analytics today.