PKU-Alignment/align-anything
A modular framework for aligning multi-modal large language models with human feedback using RLHF, DPO, and PPO techniques.

Align-Anything is an open-source framework for training all-modality large language models (any-to-any models) to align with human intentions and values. It provides modular implementations of alignment algorithms including SFT, DPO, PPO, and O1-like training approaches. The framework supports fine-tuning diverse multi-modal models across image, video, audio, and text modalities, offering a unified CLI for experimenting with different alignment methods.