The environment is the hard part now, and here's the proof

What it does

FineEnvs collects self-contained projects that each take an RL environment from design to a trained model: LaTeX OCR with a verifiable reward, a VLM learning geolocation from Mapillary panoramas, watercolour painting scored by an aesthetic preference model, 5,394 verified data-analysis tasks, multilingual OCR/ASR corpora, and port-logistics simulations graded against a CP-SAT-proven optimum. Everything — environments, datasets, models, demos — is published on Hugging Face, and the claim is that every rollout shown has actually been executed.

The interesting bit

Project 00 implements the same three environments six times across openenv, verifiers, skyrl_gym and friends — the diff between two server.py files is the lesson in framework dialects. And the multilingual project documents a genuine TRL bug: the trainer drops audio from the forward pass its loss comes from, with the fix and numbers shown.

Key highlights

• Eight numbered projects, each with its own environments, notebooks, results and Hub repos
• GeoGuesser: Qwen3.5-4B beats gpt-5.4-mini and claude-haiku-4.5 on a 200-task held-out split, for about $100 of training
• SmolDataEnvs grading uses exact comparison — no LLM sits in the reward path, and every task was solved in a sandbox before admission
• Multi-harness experiments vary who runs the agent loop (TRL, OpenCode, Harbor) while keeping a fixed pass@1 eval
• Several environments are live as playable Spaces, including the LaTeX OCR reward server

Caveats

• Projects 04, 05 and 07 are marked work-in-progress in the index
• It’s a curated set of tutorials and artifacts rather than a library you import — the value is in reading and running the projects, not in a package API

Verdict

Anyone training LLM agents with RL — or confused by the flood of environment frameworks — will get real mileage out of this. If you wanted a drop-in environment SDK, this isn’t that, and it doesn’t pretend to be.