Gradients for Already-Optimized LLVM and MLIR

What it does Enzyme is a plugin that performs reverse-mode automatic differentiation on statically analyzable LLVM and MLIR. You mark a function with __enzyme_autodiff, run the Enzyme transformation pass, and the compiler substitutes the call with the gradient of the first argument. Because it works on compiler intermediate representation, it can differentiate code produced by C, C++, Rust, Julia, or any language that lowers to LLVM.

The interesting bit Most autodiff tools operate on source code or high-level computation graphs; Enzyme differentiates code after the optimizer has already run. The project claims this approach lets it meet or exceed the performance of state-of-the-art AD tools, and it handles parallel constructs—CUDA, ROCm, OpenMP, MPI, Julia tasks—by analyzing them directly in IR.

Key highlights

• Targets optimized LLVM/MLIR instead of source-level ASTs.
• Supports GPU kernels (CUDA/ROCm), OpenMP, MPI, and Julia tasks.
• Provides Julia and Rust bindings.
• Claims performance that meets or exceeds state-of-the-art AD tools.
• Peer-reviewed backing from NeurIPS 2020 and SC 2021/2022.

Caveats

• Requires statically analyzable IR, so dynamic or opaque control flow may be unsupported.
• The README is thin on integration details beyond a basic C example, deferring to an external website.

Verdict Worth exploring if your C++, Rust, or Julia pipeline compiles through LLVM and you want autodiff as a compiler pass. If you live in Python or dynamic-language frameworks that never see LLVM IR, this is not your tool.