← all repositories
Nehanth/pooled

Your phone holds layers 43–63: LLM pooling across browser tabs

Pooled splits one open model across every device in a WebRTC room, so a laptop, a desktop and a phone together run a 35B model none of them could hold alone.

pooled
Too new to rank. Velocity needs a few days of history.
not enough history yet
star history

What it does Pooled runs one open model across all the devices in a browser “room.” Each device — laptop, desktop, phone — holds a slice of the layers and runs them on its own GPU with its own WGSL kernels, while a few kilobytes of hidden state hop tab to tab over WebRTC for every token. You can chat with the result, or switch to Code mode and let it build and fix a small app in a sandboxed preview. Nothing to install, no account, and no server does any of the thinking.

The interesting bit Only the hidden state travels — 10 KB per token on the 27B, 4 KB on the MoE — never weights or activations, which is the entire reason layer-splitting over ordinary connections is viable. And since plain decode pays one network delay per device per token, speculative decoding isn’t a luxury here: at 20–50 ms of emulated latency it runs 1.6–2.1x plain on the MoE (1.5–1.8x on the 27B), and golden tests verify the speculative stream matches plain decoding.

Key highlights

  • Three models today: Qwen 3.8 27B (~17 GB across the room), Qwen 3.6 35B MoE (~22.5 GB, decodes several times faster with 8 of 256 experts active per token), and Qwen3 1.7B for rooms of phones and light laptops.
  • Each device downloads only its own layers — from Hugging Face or peer-to-peer from a device that already has them — and caches them; when someone leaves, the host re-deals their layers with one click.
  • Code mode runs a small agent loop: it writes files, serves them on a virtual localhost in a sandboxed preview, reads its own console errors, and fixes them — everyone in the room sees the same files and preview.
  • A room can expose an OpenAI- and Anthropic-compatible API for tools like Continue, Open WebUI, or LiteLLM; the bridge joins as a client with no layers and listens on localhost only.
  • Refreshingly honest benchmarks: they publish llama.cpp numbers they lose, with a bench log of commits and caveats.

Caveats

  • Native llama.cpp still wins plain decode (~2x on the MoE, 1.3x on the 27B), and prefill is the biggest gap — the first token of a 172-token prompt takes ~1.0 s vs ~0.07 s native on the MoE.
  • The multi-device tables come from emulated latency on loopback, all devices sharing one GPU, with no bandwidth limits, loss, or jitter.
  • Serve API v1 is chat only — tool calls, images, and JSON mode get a clear 400 — and prompts are visible to the host, and possibly the whole room.

Verdict If you have a laptop, a desktop, and a phone with idle GPUs, this is the most painless way to make them run one big model together — and the architecture docs are worth reading even if you never open a room. If you need native throughput, llama.cpp still wins.

Frequently asked

What is Nehanth/pooled?
Pooled splits one open model across every device in a WebRTC room, so a laptop, a desktop and a phone together run a 35B model none of them could hold alone.
Is pooled open source?
Yes — Nehanth/pooled is open source, released under the MIT license.
What language is pooled written in?
Nehanth/pooled is primarily written in JavaScript.
How popular is pooled?
Nehanth/pooled has 640 stars on GitHub.
Where can I find pooled?
Nehanth/pooled is on GitHub at https://github.com/Nehanth/pooled.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.