Auto-publish model metrics to pull-request threads

What it does

CML is a CLI that wires machine-learning workflows into your existing CI/CD provider. It orchestrates training and evaluation jobs inside GitHub Actions, GitLab CI, or Bitbucket pipelines, then packages metrics and plots into markdown reports that surface as pull-request comments or checks. It also provisions cloud runners on AWS or Azure when standard CI hardware is not enough.

The interesting bit

Rather than standing up a separate MLOps platform, CML treats Git and your CI provider as the experiment ledger. By pairing with DVC for data versioning, it can diff metrics and confusion matrices across commits and publish the results exactly where code review happens.

Key highlights

• Posts markdown reports with images and metrics directly to PRs on GitHub, GitLab, or Bitbucket
• Provisions self-hosted or cloud runners (e.g., AWS EC2, Azure) for heavy training jobs
• Pairs with DVC to pull external data and compare experiment history across commits
• Ships as pre-built Docker images bundling Node, Python, and DVC
• Creates pull requests automatically via cml pr create when workflows produce new artifacts

Caveats

• GitLab requires a personal access token; the README explicitly warns this is mandatory for the example workflows
• Bitbucket support is referenced but far less documented than the GitHub-centric examples
• Some command names in prose (e.g., cml send-comment) do not match the current CLI table (cml comment create), suggesting recent renaming that may trip up copy-paste explorers

Verdict

Data scientists tired of manually sharing metrics will appreciate the automation; teams already invested in a full MLOps platform may find it overlaps with existing tooling.