Text-Condition Any PyTorch Model by Adding a Decorator

What it does

This library gives you pre-built text conditioning modules for PyTorch. You pick text encoders like T5 or OpenCLIP, and it returns callable conditioning functions—using FiLM or cross-attention—that you apply to your model’s hidden layers. A class decorator can automate the plumbing entirely, letting you pass raw text strings straight into a model’s forward pass during training and inference.

The interesting bit

The decorator is the real trick: it intercepts your model’s initialization and forward method to inject conditioning functions at the layers you specify, without a ground-up rewrite. It also supports stacking multiple text embedding models—say, T5 plus CLIP—to match the conditioning approach used in eDiff-I.

Key highlights

• FiLM and cross-attention conditioning on arbitrary hidden dimensions.
• Combines multiple text encoders (e.g., T5 and OpenCLIP) for richer guidance.
• Class decorator automates wiring; train with text strings, run inference with a guidance scale.
• Built-in conditional dropout for classifier-free guidance training.
• Exposes conditioning functions directly if you prefer manual insertion over magic.

Caveats

• The magic decorator is explicitly marked as a work in progress.
• The author notes that a recent paper may have already obsoleted vanilla classifier-free guidance.
• Stress testing on complex architectures like spacetime UNets is still pending.

Verdict

Worth a look if you are building or retrofitting a generative model and want text conditioning without boilerplate. Skip it if you need a battle-tested, framework-grade solution or if you already get this for free in your diffusion codebase.