A PyTorch Sampler That Forces Your Model to Notice Rare Classes

What it does

ImbalancedDatasetSampler is a drop-in replacement for PyTorch’s default sampling strategy. It inspects your dataset’s class distribution and adjusts the probability of drawing each sample so rare classes appear more frequently in training batches and common ones less so. You get balanced mini-batches each epoch without generating a separate, resampled dataset on disk or in memory.

The interesting bit

Rather than statically duplicating minority images—a naive oversampling trick the README warns can cause overfitting—the sampler changes what the DataLoader serves by estimating inverse-frequency weights automatically. The underlying dataset stays untouched; only the view of it during training shifts.

Key highlights

• Rebalances classes dynamically during sampling instead of creating a new balanced dataset.
• Estimates per-class sampling weights automatically based on class frequencies.
• Drops into any standard PyTorch DataLoader through the sampler argument.
• The included MNIST example shows improved test accuracy for under-represented digits (2, 6, 9) without degrading performance on the majority classes.
• Designed to pair with data augmentation to further mitigate overfitting from oversampling.

Verdict

Worth a look if you are training PyTorch classifiers where a few categories dominate the loss landscape and you want a quick, non-destructive fix. Skip it if your class distributions are already roughly even or if you need fine-grained, per-sample weighting rather than per-class rebalancing.