conceptofmind/LaMDA-rlhf-pytorch
Open-source PyTorch implementation of Google's LaMDA language model with RLHF training.
Not currently ranked — collecting fresh signals.
star history
This repository provides a pre-training implementation of Google’s LaMDA model architecture in PyTorch. It replicates key architectural features including T5 relative positional bias, gated GELU activations, and a decoder-only autoregressive design. The project incorporates reinforcement learning from human feedback (RLHF) to enable training similar to ChatGPT, alongside HuggingFace dataset integration for efficient data loading and a GPT-2 compatible tokenizer.
Frequently asked
- What is conceptofmind/LaMDA-rlhf-pytorch?
- Open-source PyTorch implementation of Google's LaMDA language model with RLHF training.
- Is LaMDA-rlhf-pytorch open source?
- Yes — conceptofmind/LaMDA-rlhf-pytorch is open source, released under the MIT license.
- What language is LaMDA-rlhf-pytorch written in?
- conceptofmind/LaMDA-rlhf-pytorch is primarily written in Python.
- How popular is LaMDA-rlhf-pytorch?
- conceptofmind/LaMDA-rlhf-pytorch has 467 stars on GitHub.
- Where can I find LaMDA-rlhf-pytorch?
- conceptofmind/LaMDA-rlhf-pytorch is on GitHub at https://github.com/conceptofmind/LaMDA-rlhf-pytorch.