四款 AI Harness 对比，GPT-6 Astra Pro 当裁判

Alican Kiraz @AlicanKiraz0原推发布于 2026.10.20

在 X 上打开原推

Hi everyone, this time, I compared the outputs and performance of some of the most widely used AI Harness tools. I evaluated Pi, OMP, Droid, and Codex CLI using the same development prompt in the GLASS CATHEDRAL — Fracture Dynamics Benchmark scenario. 🔥 I used GPT-6 Luna with high effort across all tools. Instead of moving directly into development, I followed a planning-first, development-second workflow. For Pi, I explicitly instructed it to prepare a plan first, then proceeded with the development command. Task Summary; I tasked the tools with developing an application that simulates three Gothic stained-glass windows, incorporating 150–200 independent triangular glass fragments, XPBD-based breakable constraints, physical collisions, wind effects, and structural support loss. The application had to run in a single HTML file, complete a 75-second autonomous scenario, and generate a measurement-based results report. Note: I used GPT-6 Astra Pro as the Judge LLM for the evaluation. AI Harness Delivery Performance Scores /10 1st — Droid: 9.33/10 @droid 2nd — Pi: 6.43/10 @pidotdev 3rd — Codex CLI: 5.86/10 @OpenAIDevs 4th — OMP: 3.10/10 @stencil_labs Cost and Speed - Droid — $0.17 — 15 minutes - Pi — $0.15 — 42 minutes - Codex CLI — $0.34 — 24 minutes - OMP — $0.83 — 1 hour 17 minutes Results; 👉Best speed-to-cost balance and fastest completion — Droid. Despite costing just $0.02 more than the cheapest option, it completed the task in the shortest time. Its delivered code also included phase-based time series, event logs, and fracture causes in the report, which was a positive feature in terms of auditability. 👉 Lowest direct cost — Pi. However, its $0.15 cost advantage came with a completion time of 42 minutes. Therefore, in this run, the cheapest option did not provide the strongest overall efficiency. 👉 Second-fastest completion — Codex CLI. It completed the task in 24 minutes. However, it took 9 minutes longer than Droid and cost twice as much. 👉 Longest completion time and highest cost — OMP. At 77 minutes and $0.83, it ranked last in the economic dimension of this evaluation. This result does not mean that OMP's overall capabilities are insufficient, but rather that its time and cost in this task were not justified by the metrics I selected. One difference was particularly noteworthy: Droid completed the task 27 minutes faster than Pi for just $0.02 more.Compared with Codex CLI, it delivered the task in 37.5% less time and at 50% lower cost. In this benchmark, Droid clearly stood out by combining speed with low cost. Pi was the most economical option, while Codex CLI ranked second in completion time. OMP, despite taking longer and costing more in this run, ranked last in the delivery performance scores.