Menu

Benchmark · run 2026-11-21

Round 1: Claude Code as a browser agent

We gave Claude Code — running headless, with nothing but a browser — 10 tasks that each have one checkable answer, and ran a hand-written Playwright script on the same tasks as a baseline.

Scope and disclosure. This round tests one agent setup (Claude Code + Playwright MCP) with three Claude models. It is not a cross-vendor comparison, and it does not yet include Browser Use, Stagehand or Skyvern. Claude Code was run on a Claude subscription; the cost figures below are the API-equivalent cost that Claude Code itself reports for each run (total_cost_usd), not an invoice. BrowseBench has no commercial relationship with Anthropic.
Short answer: Haiku 50/50, Sonnet 50/50, Opus 30/30 correct, against 50/50 for the script. On these simple tasks the agent's median time per task was 11.1 s / 12.7 s / 12.9 s versus 1.3 s for the script, and its reported cost per task ranged from $0.004 to $0.104. A perfect score on 10 easy tasks says the setup works; it doesn't say it will on harder sites — that is what the next rounds are for.

Results

SetupCorrect answersMedian time per taskMedian turnsReported cost per task
Scripted Playwright (no model) 50/50 1.3 s — $0
Claude Code + Haiku
claude-haiku-5-5
50/50 11.1 s 6 $0.004
Claude Code + Sonnet
claude-sonnet-5-5
50/50 12.7 s 4 $0.048
Claude Code + Opus
claude-opus-5-5
30/30 12.9 s 5 $0.104

"Correct" means the final ANSWER: line matched the ground truth. Time is wall-clock per run, including the agent starting its browser; turns are Claude Code's own num_turns. Total reported cost for all agent runs in this round: $5.69.

By task

TaskScripted Playwright (no model)HaikuSonnetOpus
login-confirm
Go to https://quotes.toscrape.com/login and log in (the demo accepts any username and password). After logging in, what is the text of the link shown at the top right of the page?
Expected: Logout
5/5
1.3 s
5/5
11.6 s
5/5
11.7 s
3/3
12.8 s
js-first-author
Open https://quotes.toscrape.com/js/ (the quotes are rendered with JavaScript). Who is the author of the first quote on the page?
Expected: Albert Einstein
5/5
0.9 s
5/5
9.6 s
5/5
8.7 s
3/3
9.1 s
scroll-100th-author
Open https://quotes.toscrape.com/scroll, which loads more quotes as you scroll. Who is the author of the 100th quote on that page?
Expected: George R.R. Martin
5/5
2.1 s
5/5
16.8 s
5/5
19.7 s
3/3
22.3 s
distinct-authors
quotes.toscrape.com spreads its quotes over several pages (follow the "Next" links starting at https://quotes.toscrape.com/). How many distinct authors appear across all pages? Answer with a number.
Expected: 50
5/5
3.3 s
5/5
10.5 s
5/5
12.2 s
3/3
13.4 s
einstein-count
Across all pages of https://quotes.toscrape.com/ (follow "Next"), how many quotes are by Albert Einstein? Answer with a number.
Expected: 10
5/5
3.5 s
5/5
12.1 s
5/5
12.8 s
3/3
13.1 s
travel-cheapest
On https://books.toscrape.com/ open the "Travel" category. What is the price of the cheapest book in that category? Answer with the number only, in pounds.
Expected: 23.21
5/5
0.9 s
5/5
10.9 s
5/5
9.8 s
3/3
11.0 s
mystery-count
On https://books.toscrape.com/ open the "Mystery" category (it may have more than one page). How many books are in the Mystery category in total? Answer with a number.
Expected: 32
5/5
1.3 s
5/5
12.1 s
5/5
10.4 s
3/3
12.6 s
book-upc
On https://books.toscrape.com/ find the book "A Light in the Attic" and open its product page. What is its UPC?
Expected: a897fe39b1053632
5/5
1.3 s
5/5
10.2 s
5/5
11.3 s
3/3
12.8 s
table-max-due
Open https://the-internet.herokuapp.com/tables. In "Example 1", which person has the largest "Due" amount? Answer with that person's email address.
Expected: [email protected]
5/5
1.0 s
5/5
8.2 s
5/5
9.7 s
3/3
10.2 s
dynamic-loading
Open https://the-internet.herokuapp.com/dynamic_loading/2, click "Start" and wait for the result to appear. What text appears?
Expected: Hello World!
5/5
6.4 s
5/5
15.5 s
5/5
15.0 s
3/3
17.4 s

Setup

What this does and doesn't tell you

Download and re-run

Tasks, runner, baseline, ground-truth script and every run's raw record are in the public repository github.com/RichardDu1/browsebench-harness (MIT).