ApprenticeBench: GUI agents beat API success rates
A new benchmark, ApprenticeBench, tests computer use on a realistic accounting job, where Fable 5.1 scores 72% via GUI vs 70% via API and GPT-6 Astra 68% vs 65%, which the author says means the "CUA tax" has disappeared.
The strongest agents can now succeed more often through GUIs than through APIs. We see this on ApprenticeBench, the first benchmark for computer use and continual learning in a realistic accounting job. How well do they learn on the job and process bills through GUIs vs. APIs? - Fable 5.1: 72% vs. 70% - GPT-6 Astra: 68% vs. 65% It’s fair to say the “CUA tax” has disappeared for these models in terms of task success over a 100-task run of learning and execution. —— Some more personal takes to share: I’ve long felt, and often told fellow CUA researchers, that computer use has been seriously underestimated and evaluated too narrowly since around Opus 4 and Sonnet 4.5. A few highlights of our evaluation: - A realistic construction accounting role inside a real business app. - Extensive company knowledge and conventions to learn from historical records and online feedback. - The same frontier harnesses I use every day, rather than a toy CUA loop: Claude Code / Codex + computer-use tools + Bash + Python + filesystem + PDF tools + … - A comprehensive sweep across models, revealing striking gaps across model families (proprietary vs. open-weight), generations (Astra and Fable vs. earlier), and reasoning efforts (max to none). Some of my favorite moments are watching agents quickly master a complicated business app through a few interactions, start batching clicks and typing after a few tasks, and retain and apply lessons about both the company workflow and the visual interface using plain textual Markdown notes. Very proud to have contributed to ApprenticeBench. I’m even more excited and confident than before to build CUAs that can learn faster and get work done more efficiently and reliably.
This post quotes another post: “ApprenticeBench: GPT-6 Astra learns on the job and beats human professionals”