GDPx by SemiAnalysis: Why top AI models solve only 12% of real tasks

SemiAnalysis published a new agentic benchmark named GDPx on November 2, designed to evaluate frontier AI models on high-value professional tasks. Authors Max Kahn, Andrew Megalaa, and Dylan Patel built the system to rigorously test capabilities in domains where businesses are willing to pay for results.

• The benchmark spans 11 complex domains, including chip design, cybersecurity, law, and finance.
• Anthropic's Claude Opus 5.5 achieved the highest overall score among tested models.
• Models were given up to eight hours per task inside their native software wrappers.
• Built-in safety classifiers were shown to penalize models with lower scores due to false triggers.

Scope of Domains and Strict Methodology

GDPx covers 11 key fields: chip and hardware design, cybersecurity, biology, finance, law, quantum trading, materials science, software engineering, recursive AI self-improvement, and alignment with human goals. These areas were selected based on expected AI acceleration rather than GDP share. Fourteen partners created RL environments for the tests, allowing models up to eight hours per task within native shells like Claude Code, Codex, and Antigravity.

The authors emphasized strict quality control, criticizing non-interactive benchmarks like OpenAI's GDPval and high error rates found in Terminal Bench 4.0. By allowing models to operate in native environments, the test recreated authentic professional challenges across diverse industries, though finance and law remained undefeated by any AI.

Category Leaders and Severe Test Failures

Claude Opus 5.5 emerged as the overall leader, winning quantum trading and hardware design. OpenAI's GPT-6 Astra took the lead in self-improvement and materials science, Google's Gemini 4 Carbon triumphed in biology and software engineering, while Meta's Muse Wonder Full dominated cybersecurity. Meanwhile, finance and legal tasks saw zero successful completions across dozens of runs.

The numbers highlight extreme difficulty: in legal redlining tasks, Opus 5.5 scored an average partial grade of 44%, while financial earnings quality analysis failed in all 25 attempts. Furthermore, Opus and Astra lost points because their safety classifiers triggered on cybersecurity and biology tasks, proving that safety guardrails carry a direct penalty in percentage points.

Software Engineering Failures and Industry Takeaways

An analysis of software development errors revealed that 76% of frontier model failures stemmed from requirements outside the core prompt: breaking existing behavior, failing integrations, and ignoring edge cases. Models frequently write tests confirming their own code rather than checking if it matches the original user requirements.

For businesses and developers, these findings indicate that current autonomous agents require rigorous integration testing and oversight rather than blind trust. Whether GDPx becomes a universally adopted industry standard or remains a proprietary metric depends on whether its private task set gains broader laboratory acceptance.