Colorizing grayscale with dual decoders and color tokens

What it does DDColor is the official PyTorch implementation of an ICCV 2023 paper from Alibaba’s DAMO Academy. It automatically colorizes black-and-white photographs—historical or otherwise—and can recolor anime game landscapes into realistic styles. The system extracts multi-scale visual features to drive what the authors call “learnable color tokens,” which effectively act as dynamic color queries for the image.

The interesting bit The architecture splits colorization across dual decoders that jointly optimize these color queries. Rather than having a single network hallucinate hues, the two decoders work in tandem, which the authors credit for their claimed state-of-the-art results on automatic colorization.

Key highlights

• Four pretrained checkpoints available, including a lightweight ddcolor_paper_tiny variant
• Runs via Hugging Face Hub, ModelScope, Replicate, or fully offline with ONNX export
• Local inference does not require the BasicSR training framework
• Bundles a Gradio demo for quick browser-based tests
• Training pipeline borrows from BasicSR and incorporates components from ConvNeXt, Mask2Former, and DETR

Verdict A solid choice for researchers or developers who want ready-made photo colorization with multiple deployment options. Less appealing if you need a training pipeline that works without first downloading separate ConvNeXt and InceptionV3 weights.