luyug/GradCache
A memory-efficient training technique for scaling contrastive learning batch size far beyond GPU/TPU constraints using gradient caching.
Not currently ranked — collecting fresh signals.
star history
Gradient Cache enables training contrastive learning models with arbitrarily large batch sizes on limited hardware by caching gradients and only keeping one model copy in memory at a time. It supports both PyTorch and JAX/Flax frameworks, making it adaptable across deep learning ecosystems. The technique was developed for dense passage retrieval (DPR) systems and embedding training, which are core components of RAG pipelines and LLM application stacks.
Frequently asked
- What is luyug/GradCache?
- A memory-efficient training technique for scaling contrastive learning batch size far beyond GPU/TPU constraints using gradient caching.
- Is GradCache open source?
- Yes — luyug/GradCache is open source, released under the Apache-2.0 license.
- What language is GradCache written in?
- luyug/GradCache is primarily written in Python.
- How popular is GradCache?
- luyug/GradCache has 443 stars on GitHub.
- Where can I find GradCache?
- luyug/GradCache is on GitHub at https://github.com/luyug/GradCache.