Stable Diffusion distilled into single-digit steps

What it does

Latent Consistency Models (LCM) is the official release for two research papers that train existing Stable Diffusion checkpoints—DreamShaper, SD-XL, SSD-1B—to converge in as few as four inference steps. The project provides model weights, training scripts for consistency distillation, and an official pipeline and scheduler inside the Hugging Face diffusers library. It handles both text-to-image and image-to-image generation, with support for NVIDIA GPUs, Apple Silicon, and Intel XPU.

The interesting bit

The real kicker is LCM-LoRA: adapter weights that accelerate off-the-shelf Stable Diffusion models without a full parameter fine-tune. By distilling classifier-free guidance directly into the model input, the sampler internalises what normally requires separate computation at every step, cutting the busywork down to a single-digit loop.

Key highlights

• Official pipelines and schedulers live inside Hugging Face diffusers, so it is not a fringe fork.
• Supports both txt2img and img2img, with community ports for ComfyUI, SD-WebUI, C#/ONNX Runtime, and CPU inference.
• Releases include full parameter-tuned models and LoRA adapters for SD-XL, SSD-1B, and SD-V1.5.
• Training scripts for consistency distillation are available if you want to roll your own LCM instead of using the pre-baked weights.

Caveats

• Using torch.float16 to save VRAM explicitly “may compromise image quality,” so expect a memory-quality trade-off.
• Local image-to-image demos are marked “on the way”; the bundled local Gradio setup only covers text-to-image for now.

Verdict

Worth a look if you are building real-time image tools on top of Stable Diffusion and want drop-in speed-ups. Skip it if you need a fully self-contained training suite or a polished native application; this is a research release that expects you to integrate via diffusers or third-party UIs.