Image generation with 1.58-bit weights, no cloud or WSL2 required

What it does

This repository is largely glue code around a heavily quantized 4B-parameter diffusion model. Shell and PowerShell scripts download 1-bit or 1.58-bit (ternary) weights, vendor in a FastAPI backend with a Next.js studio UI, and dispatch to platform-specific inference: MLX on Apple Silicon, gemlite/HQQ on Linux, and—unusually—native triton-windows on NVIDIA GPUs without WSL2. You get a local studio on :8000/:3000 or one-shot CLI generation against the downloaded models/ tree.

The interesting bit

The setup scripts default to a supply-chain security posture that would make most CI pipelines blush: they refuse to install any Python or npm package published less than seven days ago, as a hedge against undetected compromises. On the model side, the ternary (1.58-bit) variant is the recommended sweet spot between the extreme compression of 1-bit weights and usable output quality.

Key highlights

• Supports 1-bit and 1.58-bit (ternary) quantized model variants, with ternary recommended for better quality
• Runs on Apple Silicon (MLX), Linux NVIDIA GPUs (gemlite/HQQ), and native Windows NVIDIA GPUs (triton-windows, no WSL2)
• Includes a full local studio: FastAPI backend plus Next.js frontend, plus CLI wrappers for one-shot or warm-server generation
• Setup scripts block packages newer than 7 days by default (BONSAI_PACKAGE_MIN_AGE_DAYS=7) as a supply-chain guardrail
• 503 stars, actively documented with platform-specific setup notes and a Windows FAQ for common failure modes

Caveats

• Windows setup has documented rough edges: PowerShell execution policies, vcredist dependencies, old NVIDIA drivers, and OOM errors at 1024×1024 are all listed in the known-failure FAQ
• The one-shot generate.sh pays a cold-start penalty (model load plus JIT compilation) on every invocation; the docs explicitly recommend using the studio server and send_request.sh instead

Verdict

Worth a look if you want private, local image generation on consumer hardware—especially if you are on Windows and tired of WSL2 gymnastics. Skip it if you need production-grade image quality or a mature, battle-tested ecosystem; this is a demo built around an experimental, heavily compressed 4B model.