policy-gradient/GRPO-Zero
A minimal implementation of GRPO (Group Relative Policy Optimization) for training large language models with reinforcement learning.
Not currently ranked — collecting fresh signals.
star history
This repository implements DeepSeek R1’s GRPO training algorithm from scratch, supporting token-level policy gradient loss and improvements from the DAPO project. It removes the KL divergence term and value estimation network to reduce GPU memory usage, allowing training on a single A40 GPU with 48GB VRAM. The implementation only depends on tokenizers and PyTorch, avoiding transformers and vLLM dependencies.
Frequently asked
- What is policy-gradient/GRPO-Zero?
- A minimal implementation of GRPO (Group Relative Policy Optimization) for training large language models with reinforcement learning.
- Is GRPO-Zero open source?
- Yes — policy-gradient/GRPO-Zero is open source, released under the Apache-2.0 license.
- What language is GRPO-Zero written in?
- policy-gradient/GRPO-Zero is primarily written in Python.
- How popular is GRPO-Zero?
- policy-gradient/GRPO-Zero has 1.9k stars on GitHub.
- Where can I find GRPO-Zero?
- policy-gradient/GRPO-Zero is on GitHub at https://github.com/policy-gradient/GRPO-Zero.