Beyond LoRA: a parameter-efficient tuning zoo for diffusion

What it does

LyCORIS is a Python library that wraps multiple parameter-efficient fine-tuning algorithms—LoRA, LoHa, LoKr, (IA)³, DyLoRA, and native dreambooth-style tuning—into a single toolkit aimed at Stable Diffusion. It provides standalone wrappers that can be attached to arbitrary PyTorch modules, plus integration modules for popular training stacks like kohya-ss. The project also ships utility scripts to extract adapters from full models, merge them back with optional high-precision weight snapshots, and convert between training and inference formats.

The interesting bit

The library treats adapter stacking as a first-class feature: you can layer multiple LycorisNetwork wrappers on the same module, remove one without disturbing the others, and restore original weights exactly using an opt-in precise-merge mode. That is the unglamorous but essential plumbing that keeps multi-adapter workflows from drifting into numerical chaos.

Key highlights

• Implements a whole zoo of methods (LoRA/LoCon, LoHa, LoKr, (IA)³, DyLoRA) with a documented trade-off matrix for fidelity, flexibility, diversity, and size—though the authors warn the table should be taken with a grain of salt.
• Standalone wrappers work on any PyTorch module, not just U-Nets, and the author has used them to fine-tune Phi-1.5 language models in a separate project.
• Supports high-precision merge and unmerge operations to avoid numerical drift when baking multiple adapters sequentially into a base checkpoint.
• Ships conversion scripts for HCP-Diffusion and sd-webui formats, plus bundle-format handling for pivotal tuning workflows.

Caveats

• Third-party support (ComfyUI, InvokeAI, CivitAI, etc.) lags behind new algorithm types; the README explicitly asks users to beg those developers for updates.
• HCP-Diffusion integration was dropped in v3.0.0 and is waiting on the upstream project to catch up.
• The comparison table of methods is self-described as rough and highly dependent on dataset, task, and hyperparameters.

Verdict

Worth a look if you are actively training or merging Stable Diffusion adapters and want to experiment with algorithmic trade-offs beyond vanilla LoRA. Skip it if you just need a single, well-supported inference plugin and do not care about the training mechanics.