vacancy/Synchronized-BatchNorm-PyTorch
A PyTorch module implementing synchronized batch normalization that computes statistics across all devices during distributed training.
Not currently ranked — collecting fresh signals.
star history
This repository provides a synchronized batch normalization implementation that differs from PyTorch’s built-in BatchNorm by reducing mean and standard deviation across all devices rather than per-device. This ensures accurate batch statistics when training with multiple GPUs using DataParallel. The implementation is particularly important for tasks with small per-GPU batch sizes, such as object detection, where standard batch normalization can degrade performance. For single-GPU or CPU-only scenarios, it behaves identically to PyTorch’s built-in implementation.
Frequently asked
- What is vacancy/Synchronized-BatchNorm-PyTorch?
- A PyTorch module implementing synchronized batch normalization that computes statistics across all devices during distributed training.
- Is Synchronized-BatchNorm-PyTorch open source?
- Yes — vacancy/Synchronized-BatchNorm-PyTorch is open source, released under the MIT license.
- What language is Synchronized-BatchNorm-PyTorch written in?
- vacancy/Synchronized-BatchNorm-PyTorch is primarily written in Python.
- How popular is Synchronized-BatchNorm-PyTorch?
- vacancy/Synchronized-BatchNorm-PyTorch has 1.5k stars on GitHub.
- Where can I find Synchronized-BatchNorm-PyTorch?
- vacancy/Synchronized-BatchNorm-PyTorch is on GitHub at https://github.com/vacancy/Synchronized-BatchNorm-PyTorch.