A benchmark where agents rebuild real websites from scratch
WebsiteBench hands agents a live site and asks: can you clone it — looks and behavior — just by poking around in a browser?
What it does
Agents get access to a production-level website through a browser and must reverse-engineer a full-stack version that matches its appearance and behavior. It’s not screenshot-to-HTML — the agent has to infer data models, interactions, and styling from exploration alone. The benchmark spans 70 websites with 46,916 hidden test cases, distributed via Harbor Hub and Hugging Face.
The interesting bit
Scoring is strict: each task carries 302 to 2,016 hidden test cases, and a case only passes when functional checks succeed and every visual checkpoint hits a normalized SSIM of 0.99. That’s pixel-close reproduction plus working behavior, not just vibes. There’s also an oracle agent mode that runs the reference solution, so you can sanity-check a task before blaming your model.
Key highlights
- 70 real-world websites, from small tasks up to large ones (task tiers like
websitebench/medium@v1) - 46,916 hidden test cases — agents can’t game what they can’t see
- Dual scoring: functional checks plus SSIM ≥ 0.99 visual matching
- Runs on the Harbor framework with Docker-based environments
- Apache-2.0 code and verifier; reference sites are non-commercial research only
Caveats
- The reference websites reproduce third-party sites, so the dataset is restricted to non-commercial research use.
- The README doesn’t say which 70 sites are included or how they were selected — you’ll need the task list docs for that.
- Task definitions carry a
harbor-canaryGUID, an anti-training-contamination measure worth knowing about if you handle the data.
Verdict
Worth a look if you evaluate coding agents or want a harder, more realistic alternative to UI-generation benchmarks. If you just want a tool that clones websites for you, this isn’t that — it’s the ruler, not the carpenter.
Frequently asked
- What is TIGER-AI-Lab/WebsiteBench?
- WebsiteBench hands agents a live site and asks: can you clone it — looks and behavior — just by poking around in a browser?
- Is WebsiteBench open source?
- Yes — TIGER-AI-Lab/WebsiteBench is open source, released under the Apache-2.0 license.
- What language is WebsiteBench written in?
- TIGER-AI-Lab/WebsiteBench is primarily written in Python.
- How popular is WebsiteBench?
- TIGER-AI-Lab/WebsiteBench has 606 stars on GitHub.
- Where can I find WebsiteBench?
- TIGER-AI-Lab/WebsiteBench is on GitHub at https://github.com/TIGER-AI-Lab/WebsiteBench.