WHAT NEW IN X10.31 SAT

X 上的 AI 合集档案 · 22 个合集 · 61,986 条

gpt-6IMAGE

WindTunnel 实测:Astra 代码执行操控浏览器

Astra 用代码执行满分通关 49 项任务,但比 WebMCP 方案贵 49 倍、慢 2.9 倍。

idan levin
@0xidanlevin2026.09.14收录原推发布于 2026.09.08
WindTunnel 实测:Astra 代码执行操控浏览器
idan levin @0xidanlevin原推发布于 2026.09.08
在 X 上打开原推

Today we added GPT-6 Astra to WindTunnel, our open WebMCP benchmark. The results are quite surprising: 1/ Both GPT-5.6 Luna with WebMCP and GPT-6 Astra with code execution solve 49/49 tasks, perfect score. (code execution is Astra's recommended method). 2/ WebMCP with GPT-5.6 Luna is ~49x cheaper than GPT-6 Astra using code execution. 3/ WebMCP is also 2.9x faster and uses 4.2x fewer tokens. Full benchmark here: https://webmcp.com/benchmark A few words on how Astra operates the browser, and what these results mean: WindTunnel compare different ways an agent with a browser can operate the same website: WebMCP, screenshot-based computer use, DOM/accessibility, and code execution. For Astra, the recommended method is code execution: you send Astra the page context, it returns a code script that runs the page, you execute it and then send a screenshot of the result back. This lets Astra perform multiple actions in one step instead of going screenshot -> click -> screenshot loop for every action. Code execution allows the agent to operate the browser *really* fast (all of these cool demos you’ve been seeing). And it worked very well: Solved 49/49 tasks (a lot of the “pure” computer use methods get poor results in the benchmark). But compared with GPT-5.6 Luna using WebMCP, it was still: ~49x more expensive 2.9x slower 4.2x heavier on tokens Which means even fancy code execution doesn't beat good old APIs with a good description. In theory Astra should shine on long runs - one script can replace hundreds of clicks - so it may still win where actions are numerous and no tool covers them, like bulk edits on a site without WebMCP. But on sites that do expose tools, we saw the opposite: on our longest tasks, Astra's code execution cost over 100× more than Luna on WebMCP. The benchmark is open-source, feel free to run it yourself and update us with any feedback. If you find WebMCP interesting, we’ll be holding a free WebMCP workshop next week for both developers and enthusiasts. Feel free to sign up and find more information here: https://luma.com/d9epljk6

x.com/0xidanlevin/status/2097346723979182479采集于 2026.09.14 16:18