Generating transparent PNGs natively inside Stable Diffusion

What it does

sd-forge-layerdiffuse is a work-in-progress extension for SD WebUI (via Forge) that generates transparent images and decomposed layers directly during the diffusion process. Instead of post-processing a generated image to remove a background, it manipulates the model’s latent distribution into a “transparent latent space” and decodes it through a custom VAE pipeline, producing PNGs with actual alpha channels. The result handles semi-transparent materials like glass, glowing effects, and fine hair detail that background-removal tools typically mangle.

The interesting bit

Transparency is not a post-processing mask; it is baked into generation. The author uses rank-256 LoRA offsets and attention-sharing modules to alter how the model samples latents, enabling it to generate foreground, background, and blended compositions simultaneously for SD1.5. An equivalent joint-generation mode for SDXL remains on hold because it currently demands roughly four times the VRAM and runs three times slower.

Key highlights

• Generates native transparent images—glass, fur, whiskers, glow—that simple background removal cannot reproduce.
• Supports conditional layer workflows: derive backgrounds from foregrounds, foregrounds from backgrounds, or generate blended compositions.
• Model downloads and selections are fully automatic; most users can ignore the model zoo entirely.
• Ships lightweight VAE encoder/decoder variants that cut VRAM usage compared with the original paper’s architecture without sacrificing quality, according to the author.
• img2img support is finished.

Caveats

• The project is explicitly marked WIP, and some advanced SDXL features (like joint layer generation) are paused until inference speed and memory use improve.
• The author reports sampler-specific artifacts—DPM++ can occasionally introduce glitches that Euler A resolves—so your mileage may vary depending on which scheduler you pick.

Verdict

Worth exploring if you produce game assets, compositing elements, or stock imagery where clean alpha channels matter. Skip it if you are not already running Forge or SD WebUI, or if you need a polished, finalized tool rather than an active research extension.