LLMs that answer with probabilities, and only think when they're unsure

What it does

This is a family of open-weight models (0.8B to 27B) plus an inference engine and HTTP server for vLLM, llama.cpp and MLX. You send a state — text, JSON or images — and a set of typed questions: a choice among options, a noul yes/no judgement, or a score on a scale. You get back a probability for every option of every question, in a single forward pass. There’s also an none_probability field flagging when none of your listed options fit, which is a quietly nice escape hatch for classification tasks.

The interesting bit

The AutoThink variants only reason when the first read-out’s top probability falls below a gate (0.6 by default). Confident answers skip the chain-of-thought entirely — the demos show it steering Mario, chess, StarCraft II and even a rocket launch at one frame per second, which only works if most decisions are one cheap pass. The reasoning itself is trained with RLCD², reinforcement learning for calibrated deliberative decisions.

Key highlights

• Instant models answer in 25–52 ms median on one H200 (0.8B to 27B, one request in flight); AutoThink needs 38–59 ms when sure.
• Instant models beat the strongest comparable decision models by 7.4–16.9 points on Decision Index 0.3; AutoThink adds 11.5–18.4 points of DecisionPivot pair accuracy.
• Weights on Hugging Face in GGUF and MLX 8/4-bit, Apache-2.0 code, Python 3.9+.
• Free unlimited hosting of the 27B AutoThink on bamboonode.dev with the same request format.
• Tunable controls: think_mode, gate, think_budget per request.

Caveats

• The published AutoThink numbers use a 0.9 gate, not the 0.6 default — reproduce with care.
• Decision Index 0.3 is a public subset; the full evaluation lives in the arXiv report, which we haven’t verified.
• The vLLM plugin pins to specific versions (0.19.1 for the published numbers, 0.31 also supported).

Verdict

Worth a look if you’re building agents or routing pipelines and want calibrated probabilities instead of parsing prose. If you need open-ended generation or conversation, this is the wrong tool — it’s a decision engine, deliberately.