Tenstorrent's open-source stack for its own AI silicon

What it does

TT-Metal is two layers in one repo. TT-Metalium is the low-level C++ programming model for writing kernels on Tenstorrent hardware. TT-NN sits above it, a Python and C++ neural-network operator library that implements the usual suspects: transformers, convolutions, flash attention, and enough tensor shuffling to run full models. The README is heavy on benchmark tables and light on architecture prose, but the message is clear: this is the official path to make Tenstorrent’s chips do useful work.

The interesting bit

The project treats scale-out as a first-class citizen, not an afterthought. There are dedicated tech reports for programming meshes of devices, multi-mesh scale-out, and something called TT-Fabric that handles chip-to-chip communication. That is the unglamorous part that usually breaks in production, and they seem to know it.

Key highlights

• Ships ready-made model demos for Llama 3.3 70B, Qwen 2.5, Mixtral 8x7B, Whisper, and image/video generation models
• Exposes both a high-level Python API (ttnn) and bare-metal kernel programming in C++ with RISC-V compute cores
• Targets two hardware generations: Wormhole (current) and Blackhole (still being optimized, per the README)
• Integrates with vLLM for inference serving, though performance numbers are tagged as tt-metal specific and may differ under vLLM
• Includes a small constellation of companion tools: TT-NN Visualizer, TT-Exalens debugger, TT-SMI system interface, and Model Explorer

Caveats

• Blackhole support is explicitly flagged as “under active development,” so early adopters should expect rough edges
• Performance tables show several models missing their target T/S/U numbers, which the README presents without commentary
• The repo is tightly coupled to Tenstorrent hardware; there is no emulation path or fallback for standard GPUs

Verdict

Worth a look if you have Tenstorrent silicon in hand or are evaluating it against NVIDIA’s lock-in. Everyone else can observe from the sidelines until the hardware becomes easier to obtain.