pals-ttic/sjc
Score Jacobian Chaining lifts pretrained 2D diffusion models to generate 3D content by aggregating 2D scores through a differentiable voxel renderer.

The project proposes Score Jacobian Chaining, which applies the chain rule on learned gradient fields of diffusion models and back-propagates through a differentiable renderer. It aggregates 2D scores from multiple camera viewpoints into a 3D score, repurposing pretrained models like Stable Diffusion for 3D generation. The method addresses distribution mismatch issues that arise in this cross-dimensional lifting application.