What it does

PacNet replaces standard PyTorch convolutions, transposed convolutions, and pooling with “pixel-adaptive” versions. Each layer takes both an input tensor and a guidance tensor; the guidance reshapes the convolution kernel locally at every spatial location. The API is deliberately familiar—PacConv2d accepts most nn.Conv2d arguments and guarantees identical output sizes when guidance is ignored.

The interesting bit

The library ships with PacCRF and PacCRFLoose, which wrap mean-field CRF inference as differentiable PyTorch modules. That means you can drop a learned CRF at the end of a segmentation network and backprop through the whole thing. The README also notes that weights for unary and pairwise terms can be tensors, not just constants—so the CRF itself can be learned.

Key highlights

• Five layer types: PacConv2d, PacConvTranspose2d, PacPool2d, PacCRF, PacCRFLoose
• Guidance can be raw features or pre-computed kernels via packernel2d
• Pre-trained models and benchmarks for joint depth upsampling (NYU Depth V2), with 4×/8×/16× results against bilinear and DJIF baselines
• Supports PyTorch 0.4 through 1.1 (the README’s stated range)
• CC BY-NC-SA 4.0 license—non-commercial only

Caveats

• PyTorch version ceiling of 1.1 is now several years old; compatibility with current releases is unclear
• Kernel sizes must be odd, and padding has a strict upper bound relative to dilation
• No groups or padding_mode support in conv layers

Verdict

Worth a look if you’re building dense prediction pipelines where one modality should steer another—depth from RGB, segmentation from edges, etc. Skip it if you need standard grouped convolutions or a commercial license.