C video generator keeps two DiT blocks in RAM, streams the rest

What it does h3-metal is a from-scratch C inference engine for MiniMax-H3, a transformer-based video and audio generation model. It targets Apple Silicon through Metal, exposing both one-shot CLI generation and an interactive terminal session that keeps prompt conditioning, DiT state, and video decoders resident between generations. First/last-frame anchoring and ordered Ref2VA image references work end to end.

The interesting bit The project treats memory pressure as a first-class design constraint. Its --ssd-streaming mode retains only two DiT blocks in unified memory and reads the remaining weights from disk while the GPU computes, cutting tracked transformer storage from roughly 36.5 GiB to about 2 GiB at a 26–84 percent speed penalty. The author also validates aggressive quality shortcuts—four-step denoising, layer thinning, velocity reuse, and internal canvas downsampling—with explicit SSIM scores against a 50-step reference rather than eyeballing.

Key highlights

• Hand-written C and Metal kernels; not a Python framework wrapper.
• Interactive Iris-style session with commands like !first, !last, !ref-image, and !ssd-streaming.
• SSD streaming uses the original BF16 checkpoint without quantization and produces byte-identical output to the full-residency path.
• Four-step denoising completes in about 3.5 seconds on an M5 Max versus 26.4 seconds for the 50-step reference.
• Terminal frame preview support for Kitty, Ghostty, iTerm2, WezTerm, and Konsole.
• Optional INT8 row-wise FC2 quantization and token-reduction modes for faster iteration.

Caveats

• Apple Silicon and Metal only; there is no portability layer.
• Certain aggressive presets are incompatible—combining --layers 40, --reuse 3, and --token-reduction produced color ringing and ghosted limbs in validation.
• SSD streaming cannot be combined with --use-int8-row-fc2.
• Benchmarking is sensitive to thermal throttling; the README warns that machines need warmup for consistent numbers.

Verdict Terminal-native Mac users who want to iterate on short video clips without allocating 36 GiB of transformer weights should try this. Everyone else—especially non-Apple-Silicon developers—should wait for a port.