Relative camera geometry inside attention layers

What it does

PRoPE is a drop-in replacement for standard scaled dot-product attention that conditions on camera intrinsics and extrinsics. Instead of preprocessing absolute raymaps or relative pose embeddings, it bakes the relative projective transformation between views directly into the attention scores. The result is a geometry-aware token interaction that understands where each patch sits in 3D space relative to other cameras.

The interesting bit

The entire implementation fits in a single file for both PyTorch and JAX—no sprawling modules, no custom CUDA kernels. It accepts viewmats and Ks alongside your Q/K/V tensors, and the attention mechanism handles the rest. It is already shipping in production systems like Tencent’s Hunyuan World Model 1.5, which suggests the overhead is negligible enough for generative world models.

Key highlights

• Standalone single-file implementations in prope/torch.py and prope/jax.py
• Drop-in API intended to replace torch.nn.functional.scaled_dot_product_attention
• Claims improvement over LVSM for novel view synthesis (see nvs branch)
• Adopted by Tencent Hunyuan World Model 1.5 for camera control
• Extended by follow-up work for distorted cameras and feed-forward reconstruction

Caveats

• Stereo depth estimation experiments with UniMatch are noted as “to be released”
• The README frames the method as a study comparing absolute, relative pose, and projective encodings, so the gains may be task-dependent

Verdict

Worth a look if you are building multi-view transformers and want camera conditioning without architectural surgery. Skip it if you are not working with calibrated multi-camera setups.