Research firm SemiAnalysis has launched GDPx, a new benchmark that drops AI agents into long, realistic work tasks across 11 high-value fields, from chip design to private-equity finance. The headline result is humbling: the best model on the leaderboard, Anthropic’s Claude Opus 5.5, completes just 12% of tasks, with Google’s Gemini 4 Carbon and OpenAI’s GPT-6 Astra close behind.
That gap between a frontier model’s chatbot polish and its ability to finish hours-long professional work is exactly what the GDPx benchmark is built to expose. Here is what it measures, how the top models did, and what developers and businesses should take from it.
Key takeaways
- GDPx v1.0 contains 206 agentic tasks across 11 domains, built with 14 specialist data partners, and ranks 13 models.
- Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon (10.9%) and GPT-6 Astra (9.6%).
- Gemini 4 Carbon costs roughly a third as much per task as Opus 5.5 or GPT-6 Astra.
- Safety classifiers pulled down Opus and Astra scores in cybersecurity, biology and alignment tasks, so the overall ranking depends on which domains you count.
- Finance and legal tasks were so hard that models scored zero under strict pass/fail grading.
What happened
On November 2, SemiAnalysis published GDPx: Measuring Frontier Model Capabilities, introducing what it calls a frontier-level test of general agentic ability. Rather than asking models short questions, GDPx hands them a working environment and a goal, then lets them run for up to eight hours by default.
The 11 domains are chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. SemiAnalysis says it picked these because it expects AI revenue to concentrate where frontier intelligence can be deployed profitably first, and deliberately left out sectors such as government, real estate and insurance.
Each domain’s tasks come from a specialist partner, such as LatchBio for biology, Datacurve for software engineering and Third Moment for quant trading, and every partner had to pass SemiAnalysis’s automated quality checks. The finance track, for example, recreates a private-equity deal spread across thousands of anonymized files, rather than spelling everything out in a prompt.
The GDPx leaderboard
According to the launch post and coverage from Unite.AI and OfficeChai, the top of the table looks like this:
- Claude Opus 5.5 (Anthropic): 12.0%, about $51 per task
- Gemini 4 Carbon (Google): 10.9%, about $16 per task
- GPT-6 Astra (OpenAI): 9.6%, about $54 per task
- Muse Wonder Full (Meta): 9.0%
Domain winners vary. SemiAnalysis calls Opus 5.5 the best all-around model, winning quant trading, hardware design and chip design. GPT-6 Astra takes RSI and materials science. Gemini 4 Carbon is best in biology and narrowly wins software engineering. Meta’s Muse Wonder Full is described as the strongest cybersecurity model by a meaningful margin. SemiAnalysis says SpaceXAI, Meta and open-weight models trail the three leading labs overall.
Models ran in their own native agent harnesses where available: Claude Code for Opus, Codex for Astra and Antigravity for Gemini. The headline score is pass@1 averaged over three attempts, with every domain weighted equally.
Why it matters
Many popular AI benchmarks are close to saturated, with top models bunched near the ceiling. That makes them poor at telling buyers which model will actually finish real work. GDPx flips this: low scores are intentional, leaving plenty of headroom to measure progress over the next few model generations.
It is also a direct challenge to OpenAI’s GDPval. SemiAnalysis argues that GDPval’s tasks are one-shot and non-interactive, with all context provided up front, and that its rubric grading is noisy. GDPx instead rewards the skills agents need in production: digging through messy files, using tools, recovering from mistakes and staying on track for hours.
The safety-classifier caveat matters too. Anthropic and OpenAI both run safeguards that can refuse or interrupt work in cybersecurity and biology. SemiAnalysis says those classifiers lowered Opus 5.5 and GPT-6 Astra scores in those areas, and it publishes a view without cyber, biology and alignment. In that adjusted view, OfficeChai reports GPT-6 Astra edges ahead of Opus 5.5 by a fraction of a point while Gemini 4 Carbon falls to fifth. In other words, “best model” depends heavily on your use case.
What it means for developers and businesses
If you are building with AI agents, GDPx is a useful reality check rather than a reason to stop. A few practical lessons:
- Keep humans in the loop for long tasks. Even the strongest agent fails most hours-long professional tasks end to end. Design workflows with checkpoints, review steps and narrow scopes.
- Pick models per domain, not per leaderboard. The winner changes by field. A biology or software team may get more from Gemini 4 Carbon; hardware and quant teams may prefer Opus 5.5.
- Watch cost per task. Gemini 4 Carbon delivers about 90% of Opus 5.5’s score for roughly a third of the price. At scale, that difference can decide whether an agent product is profitable.
- The harness matters. Scores were measured inside each lab’s own coding agent. Results in your own stack, prompts and tools may differ, so run your own evaluations on real tasks.
- Expect refusals in sensitive fields. If you work in security research or life sciences, safety classifiers may block legitimate work. Plan for this when choosing a provider.
For teams already splitting work across multiple coding agents, GDPx’s domain breakdown is a reasonable starting point for routing different jobs to different models.
Limits of the benchmark
GDPx is new, and its numbers should be read with care. Version 1.0 has 206 tasks, so individual domains rest on a few dozen tasks or fewer, and small changes can shift rankings. SemiAnalysis also reports partial scores for finance and legal because binary grading produced zeros across the board, which shows how far agents remain from handling those jobs alone. Much of the per-domain detail sits in charts on the GDPx site, and independent replication has not yet been published.
What to watch next
Three things are worth tracking. First, whether labs start citing GDPx in model launches, as they now do with SWE-bench and similar tests. Second, how quickly the top score moves off 12%; with new frontier releases arriving every few months, the benchmark should show whether agentic progress is accelerating. Third, how OpenAI responds to the criticism of GDPval, its own measure of economically valuable work.
For now, the message is simple: today’s best AI agents are impressive assistants, but they are not yet ready to run a deal room or design a chip on their own.