UltraChat provides 1.57 million diverse multi-round dialogue examples designed for training chat language models. The repository releases both the dataset and trained models (UltraLM series) including a 13B and 65B variant. The models were trained on this data and achieved top rankings on the AlpacaEval benchmark. The project also includes related resources like reward models (UltraRM) and preference datasets (UltraFeedback).