EleutherAI/gpt-neox
A library for training large-scale autoregressive language models on GPUs, built on Megatron and DeepSpeed.
Not currently ranked — collecting fresh signals.
star history
GPT-NeoX is an implementation of model-parallel autoregressive transformers designed for training billion-parameter language models. It builds on NVIDIA’s Megatron-LM framework and incorporates techniques from DeepSpeed, including novel optimizations for distributed training. The library supports diverse HPC infrastructures including Slurm, MPI, and cloud platforms, and is used by researchers at academic, industry, and government labs for large-scale LLM research.
Frequently asked
- What is EleutherAI/gpt-neox?
- A library for training large-scale autoregressive language models on GPUs, built on Megatron and DeepSpeed.
- Is gpt-neox open source?
- Yes — EleutherAI/gpt-neox is open source, released under the Apache-2.0 license.
- What language is gpt-neox written in?
- EleutherAI/gpt-neox is primarily written in Python.
- How popular is gpt-neox?
- EleutherAI/gpt-neox has 7.4k stars on GitHub.
- Where can I find gpt-neox?
- EleutherAI/gpt-neox is on GitHub at https://github.com/EleutherAI/gpt-neox.