Fifteen ops, one graph, and a compiler that hunts for its own kernels

What it does Luminal compiles neural networks into static computation graphs and executes them later — nothing runs until you call execute. It ships as a statically-linked Rust crate that talks directly to CUDA and Metal, and it can also serve as a torch.compile backend for existing PyTorch models. The README claims Q8 Llama 3 8B at roughly 80% of theoretical max on an H100.

The interesting bit Everything reduces to 15 primitive ops — a RISC-style instruction set that still covers transformers and convnets. Instead of a pile of hand-written rewrite rules, Luminal searches the optimization space, which is how it claims to rediscover things like Flash Attention without anyone coding them explicitly.

Key highlights

• Dynamic shapes modeled as symbolic dimensions ((s, 4096), (b, h, w + 3)), so the compiler keeps full shape visibility while specializing aggressively.
• Devices, dtypes (mxfp4, nvfp4, fp8), and even autograd are handled at compile time via global graph knowledge.
• Kernel libraries like FlashInfer and cuBLASLt live inside the search space.
• Ops are tested against equivalent PyTorch implementations for correctness.
• The core is deliberately small — the authors claim you can read it in an afternoon.

Caveats

• The README itself flags that test coverage needs improvement (issue #20 is linked from the correctness section).
• No ROCm backend yet, and the public benchmarking suite is still on the roadmap — the 80% H100 figure is the vendor’s own claim.
• Aiming for only ~80% of the PyTorch API, so exotic ops may not translate.

Verdict Worth a look if you care about inference performance, compiler design, or want a torch.compile backend with a genuinely different architecture. Skip it if you need eager debugging, AMD GPUs, or full PyTorch API parity today.