Run FLUX on a GPU that has no business running FLUX

What it does

ComfyUI-GGUF adds custom nodes that load diffusion models in GGUF format — the same variable-bitrate quantization scheme that made large language models runnable on consumer hardware. It replaces the standard model loader with a Unet Loader (GGUF) node and includes a quantized T5 text encoder loader for extra VRAM savings.

The interesting bit

The author notes that old-school UNet models (with their conv2d layers) fell apart under quantization, but transformer-based diffusion models like FLUX and SD3.5 tolerate aggressive bit-crushing far better. This is a rare case where a format migration from an entirely different domain — LLM inference — actually ports over cleanly.

Key highlights

• Supports FLUX.1 [dev], FLUX.1 [schnell], SD3.5 Large, and SD3.5 Large Turbo in pre-quantized GGUF variants
• T5 text encoder quantization available via *CLIPLoader (gguf) nodes; loaders accept both gguf and standard safetensors/bin files
• Experimental LoRA loading via built-in ComfyUI nodes
• Includes tooling for creating custom quantizations in the tools/ directory
• Ships with a single runtime dependency: the gguf Python package

Caveats

• Explicitly marked “very much WIP” by the author
• macOS Sequoia users hit a torch 2.6.x nightly regression causing “M1 buffer is not large enough” errors; torch 2.4.1 appears to be the workaround
• LoRA loading is experimental
• Requires a recent ComfyUI build with custom ops support for UNET-only loading

Verdict

Worth a look if you’re running ComfyUI on hardware that can’t fit full-precision FLUX in VRAM. Skip if you’re on a 48GB workstation and value stability over novelty — this is still shaking out edge cases.