Transformer scaling tricks, applied to pure 3D convnets

What it does — MedNeXt is a fully 3D ConvNeXt architecture for segmenting organs and structures in volumetric medical images like CT and MRI. It packages the architecture as a standalone PyTorch module, but also provides a complete training framework built on top of nnUNet v1. The project targets the specific headaches of medical imaging: sparse annotations, large 3D volumes, and the need for memory-efficient training.

The interesting bit — Despite the paper’s title invoking transformers, MedNeXt is stubbornly convolutional—it borrows transformer-style scaling tactics and applies them to pure ConvNeXt blocks. The UpKern trick is particularly neat: you can bootstrap a large-kernel model by interpolating the weights from a small-kernel one, rather than training from scratch.

Key highlights

• Four model sizes (Small to Large) with 3×3×3 and 5×5×5 kernel variants, ranging from ~5.6M to ~63M parameters
• Gradient checkpointing at the block level to trade compute for memory when training larger models
• Standalone PyTorch blocks (MedNeXtBlock, MedNeXtDownBlock, MedNeXtUpBlock) usable outside the nnUNet pipeline
• UpKern weight initialization for transferring learned representations from small to large kernels
• Deep supervision and residual connections enabled by default for the published training configuration

Caveats

• The training pipeline is locked to nnUNet v1, and the authors note that nnUNet v2 support requires manual adaptation
• MedNeXt v1 was tested specifically with 1.0 mm isotropic spacing; using native nnUNet’s median spacing is untested and may degrade performance
• The project is under active development, so the current pipeline should be expected to shift in future versions

Verdict — Worth a look if you are working on 3D medical segmentation and want a ConvNeXt-based alternative to transformer-heavy architectures like UNETR. Skip it if you are already committed to nnUNet v2 and don’t want to port preprocessing code.