One graphics card runs a 125B model — your RAM and SSD do the rest
A 125-billion-parameter model that normally needs a server, packed onto one gaming PC — NVIDIA card plus RAM plus SSD — writing answers faster than you can read.
Strata tiers a huge mixture-of-experts model across GPU, RAM, and SSD so a server-class AI runs — briskly — on hardware you already own.

What it does Strata runs Qwen3.8-Flash-Next — a 125-billion-parameter model the README says “normally needs a server” — on an ordinary gaming PC with one NVIDIA card (12-24 GB) and 64 GB of RAM. The install is a single script that asks a few questions and fetches everything itself: engine, Python, and the ~70 GB model. Once up, it exposes OpenAI- and Anthropic-compatible endpoints on localhost, so coding agents and editors treat it as just another provider, alongside a browser app with chat and a live Monitor of the model, GPU, CPU and RAM. Thinking effort is adjustable (off/low/medium/high), image input is optional, and everything runs on your own machine.
The interesting bit The model is a team of 24,576 experts, and each token needs only about 10 of them — so nothing has to live in one place. The GPU keeps the few thousand hottest experts (and keeps learning which ones those are while you use it), RAM holds all of them, the CPU computes the misses in parallel with the GPU, and a 29 GB lookup table sits on the SSD with only a few small rows read per token. On top of that, a small built-in helper guesses the next few words and the big model checks all the guesses in one pass — speculative decoding by another name — so a single step can emit several tokens.
Key highlights
- Generation speed: the README claims 60-95 tokens/s; its own table shows 52-90 across sizes on an RTX 5070 (12 GB) + 64 GB RAM, holding 41-67 tokens/s even at 128K context, with 1,070-1,310 tokens/s prompt ingestion on a 32K prompt.
- Four compression sizes from
Q2_0(37.6 GB) toIQ3_S(54.8 GB), which the README says matches the full model on published tests; a pruned Coder variant fits 32 GB of RAM and keeps 91% of SWE-bench Verified and 99% of LiveCodeBench, per its authors. - A Swift 1.5 fine-tune thinks much shorter before answering — same per-token speed, answers arrive sooner.
- A calibration mode benchmarks a few engine settings on your actual PC and keeps the fastest; worth 7% on the Coder on the reference machine.
- Can serve your phone or other PCs on the network, gated by an API key.
Caveats
- NVIDIA-only (RTX 30/40/50, 12 GB+ VRAM), and most sizes really want 64 GB of RAM — 48 GB fits only the two smallest quants. Budget ~80 GB of disk.
- One request at a time, and the first message of a chat is read in full — about a minute per 30,000 tokens.
- The first start can freeze the whole PC for 1-3 minutes while 35-55 GB loads into RAM. The README is refreshingly upfront about this.
Verdict If you have a 12 GB+ RTX card and 64 GB of RAM and want a genuinely large model fully local behind APIs your tools already speak, this is about as turnkey as it gets. If you’re on AMD, short on RAM, or need concurrent requests, this isn’t your project — yet.
Frequently asked
- What is Niko1221/Strata?
- A 125-billion-parameter model that normally needs a server, packed onto one gaming PC — NVIDIA card plus RAM plus SSD — writing answers faster than you can read.
- Is Strata open source?
- Yes — Niko1221/Strata is open source, released under the MIT license.
- What language is Strata written in?
- Niko1221/Strata is primarily written in C++.
- How popular is Strata?
- Niko1221/Strata has 46.6k stars on GitHub and is currently accelerating.
- Where can I find Strata?
- Niko1221/Strata is on GitHub at https://github.com/Niko1221/Strata.