meta-pytorch/data
A PyTorch library providing enhanced DataLoader with checkpointing and composable streaming iterators for ML training.

Velocity · 7d
+0.7
★ / day
Trend
→steady
star history
TorchData enhances PyTorch’s data loading infrastructure with a stateful DataLoader that supports mid-epoch checkpointing via load_state_dict and state_dict methods. It also provides torchdata.nodes, a library of composable iterators for chaining dataloading and preprocessing operations in a streaming programming model. These tools are designed to make data pipelines scalable and performant for machine learning workflows.