← all repositories
daniilidis-group/neural_renderer

Differentiable 3D rendering: backprop through pixels to meshes

A PyTorch port of a CVPR 2018 renderer that lets you optimize vertices, textures, and camera pose via gradient descent.

neural_renderer
Velocity · 7d
+0.4
★ / day
Trend
steady
star history

What it does This is a PyTorch reimplementation of the Neural 3D Mesh Renderer, originally written in Chainer by the paper authors. It rasterizes 3D meshes into 2D images with a twist: the rendering pipeline is differentiable, so you can backpropagate image-space loss all the way back to mesh vertices, textures, or camera parameters.

The interesting bit Standard renderers drop gradients at the visibility boundary, which breaks end-to-end training. This one approximates those gradients, letting you do things like morph a teapot silhouette into a rectangle or recover camera pose purely through pixel comparison and gradient descent.

Key highlights

  • Ships with four working examples: multi-view rendering, vertex optimization, texture matching, and camera parameter recovery
  • Adds small API extensions over the original: general 3×4 camera matrix support, lens distortion coefficients
  • Passes all test cases from the original Chainer implementation
  • Installable via pip install neural_renderer_pytorch

Caveats

  • Locked to PyTorch 0.4.0; newer versions may hit AT_ASSERT compilation errors (fix exists on a branch, not yet merged to master)
  • “Detailed documentation will be added in the near future” — for now, the examples are your docs
  • API may change; this is a research port, not a stable library

Verdict Grab it if you’re doing differentiable rendering research or need to backprop through a rasterizer. Skip if you want production-grade 3D graphics or modern PyTorch support out of the box.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.