A saner API layer for LoRA, ControlNet, and friends

What it does

Refiners is a thin PyTorch microframework that wraps foundation models so you can train and run adapters—LoRA, ControlNet, IP-Adapter, T2I-Adapter, and others—through a unified API instead of scattered scripts. It handles SDXL, Segment Anything, DINOv2, and a growing list of diffusion and vision backbones. The project also ships ready-made implementations of recent variants like ELLA, IC-Light, HQ-SAM, and a multi-stage upscaler.

The interesting bit

The framework treats adapters as first-class citizens rather than afterthought patches, which is the entire thesis of the project. That design choice translates into a consistent API for training and running them across different foundation models.

Key highlights

• Unified adapter APIs for training and inference on top of PyTorch
• Built-in support for SDXL, ControlNet, IP-Adapter, LoRA, LCM-LoRA, SAM, DINOv2, and more
• Recent additions include ELLA prompt handling, IC-Light illumination control, MVANet segmentation, and a multi upscaler
• ComfyUI registry integration and active use in downstream projects like Finegrain Editor and Visoid
• Docs and guides cover both concepts and practical adaptation workflows

Verdict

Developers who regularly mix and match diffusion adapters should look here; purists who prefer hand-rolling every PyTorch module probably won’t.