keyu-tian/SparK
A PyTorch implementation of sparse and hierarchical masked autoencoding for pretraining any CNN backbone in a BERT-style self-supervised manner.

Velocity · 7d
+1.1
★ / day
Trend
→steady
star history
SparK applies masked autoencoder principles to convolutional networks, enabling BERT-style self-supervised pretraining on ResNet and similar architectures. The method uses sparse convolution to handle irregular masked regions and hierarchical reconstruction for multi-scale feature learning. It achieves state-of-the-art results on ImageNet classification and transfers to object detection and instance segmentation tasks.