Retrieval so fast it disappears from the latency budget

What it does

Moss is an embedded search runtime, not a hosted database. You create an index through its cloud service, load it into your application’s memory, and run hybrid semantic and keyword search—complete with built-in embeddings—in under ten milliseconds. SDKs are available for Python, TypeScript, Elixir, and C, and there is a WebAssembly build for running search directly in the browser.

The interesting bit

The architectural split is the trick: Moss Cloud manages and syncs indexes, but the query path stays entirely inside your process, or even inside the browser tab, eliminating the network round-trip that typically adds hundreds of milliseconds. The README cites a P99 of 5.4 ms on a MacBook Pro M4 against 100,000 documents, measured end-to-end including embedding generation.

Key highlights

• Hybrid semantic + keyword search with metadata filtering operators like $eq, $and, $in, and $near
• Built-in embedding models mean no OpenAI key is required, though custom models are supported
• A WebAssembly SDK (@moss-dev/moss-web) enables client-side search without any server
• Database connectors for SQLite, MongoDB, MySQL, and Supabase, plus integrations for LangChain, DSPy, LlamaIndex, Pipecat, LiveKit, and others
• Published benchmarks on an M4 Pro show a 3.1 ms P50 versus 350–600 ms for Pinecone, Qdrant, and ChromaDB under the same test conditions

Caveats

• You must sign up for Moss Cloud credentials to create and load indexes; a free tier exists, but this is not a fully self-contained open-source backend you can run entirely offline
• The benchmarks were produced on a high-end Apple Silicon Mac (M4 Pro, 24GB), so absolute latency will shift on different hardware
• Moss is explicitly a “search runtime,” not a database; persistence, clustering, and sharding are abstracted away behind the cloud service

Verdict

If you are building voice agents, real-time copilots, or any RAG pipeline where a 200 ms network hop feels like an eternity, Moss is worth a look. If you need a self-hosted vector database with full control over storage and replication, look elsewhere.