Small LLMs trained on messy data that claim to rival ChatGPT

What it does

OpenChat is a family of open-source language models—built on Llama 2 and Llama 3—that are fine-tuned with a strategy called C-RLFT, which borrows from offline reinforcement learning. The project ships model weights, an OpenAI-compatible API server built on vLLM, and evaluation tooling. The latest 8B parameter release is designed to run on a single consumer GPU.

The interesting bit

The core bet is that you can train high-performing chat models from data of wildly varying quality without hand-curated preference pairs. The team claims their 7B model not only matches an older ChatGPT on several benchmarks, but also beats Grok-0 (33B) across the board and Grok-1 (314B) on average—though, as always, benchmark tables deserve a skeptical eye.

Key highlights

• Models learn from mixed-quality data without preference labels, using a technique inspired by offline RL (C-RLFT).
• The 7B/8B models target consumer hardware: the project explicitly notes compatibility with a single RTX 3090 or 4090.
• Ships a production-oriented API server that speaks the OpenAI protocol and supports dynamic batching and tensor parallelism across multiple GPUs.
• OpenChat 3.6 (Llama 3-based) is claimed to outperform the official Llama 3 8B Instruct; earlier 3.5 scores are compared against ChatGPT (March) and Grok-1.
• Weights and evaluation pipelines are published on HuggingFace, with an online demo available.

Caveats

• Benchmark claims compare against older or proprietary baselines (ChatGPT March, Grok-0, Grok-1) using evaluation configurations the authors selected; independent verification is advisable.
• The Llama 3-based 3.6 model uses Llama 3 licensing, while the earlier 3.5 series is under Apache 2.0—check license terms before commercial use.
• The README notes that reproduction can vary due to non-zero temperature and API changes over time for MT-Bench scoring.

Verdict

Worth a look if you want a locally runnable, OpenAI-compatible chat model and are curious about training from imperfect data. Skip it if you need a fully transparent training pipeline or are allergic to bold benchmark marketing.