A decision engine that refuses to generate text

What it does

Laya is a non-autoregressive decision engine that answers typed questions—choice, score, and noul (yes/no)—over any text state such as emails, tickets, or JSON documents. It runs in a single encoder forward pass, clocked at roughly 33 ms per question on a T4 and 7.2 ms when batched. Because it never generates text, there is nothing to parse and no risk of hallucination. A built-in Router selects among three checkpoints—an English ModernBERT-large model, a 2×-faster mmBERT-base multilingual model for 100+ languages, and a dedicated typed-decisions variant—to match each request.

The interesting bit

The project treats fast “System 1” judgments as a routing and scoring problem, not a generation task. Its Router can batch heterogeneous requests that mix languages, question types, and target checkpoints into shared forward passes without changing the results, which is an unusual trick for a classifier stack.

Key highlights

• Single-pass typed decisions: choice, score, and noul.
• Automatic per-request routing across three specialized checkpoints.
• ~33 ms per question on a T4; ~7.2 ms batched; cold-start loading down to ~2 s on CPU.
• The package defers PyTorch loading until inference runs, keeping lightweight processes slim.
• Optional fast paths via TileLang, torch.compile, and ONNX Runtime, plus prediction hooks for audit, trace, cache, or redaction.
• Ships with a self-hosted HTTP server, MCP server, LangChain/LangGraph integration, and a TypeScript client.

Caveats

• Checkpoints live on Hugging Face Hub and must be downloaded on first use; fully offline operation requires preloading.
• The noul question type enforces strict true/false criteria keys, and malformed questions are rejected outright rather than silently fixed.
• The 0.3.10–0.3.11 changelogs document fixes for MPS crashes, Windows Python 3.14 loading failures, calibration overfitting, and router misrouting of European text—most appear resolved, but the breadth of edge cases suggests a wide platform surface area.

Verdict

Use Laya when you need deterministic, low-latency classification or guardrails over multilingual documents without the prompt-engineering overhead of autoregressive models. Skip it if your task requires reasoning, creativity, or open-ended text generation—this engine decides, it does not compose.