同一模型换 harness，Terminal-Bench 从 52.8% 到 66.5%
帖子讲 harness 比模型更关键，并给出 Jev 做类型化决策的成本对比：千次门控检查不到一美分。
52.8% TO 66.5%. SAME MODEL. DIFFERENT HARNESS. That is the part of AI engineering most people still refuse to understand. The model is the rented intelligence. The harness is the system you actually own. A team put the same model into a better execution loop and reportedly moved its Terminal-Bench score from 52.8% to 66.5%. They changed the system around the model: → better context → fewer, sharper tools → verification before done → hard guardrails → structured traces → fixes that survive the next run That is the compounding advantage. Most people respond to an agent failure by typing a longer instruction. Serious teams turn the failure into a permanent system change: 1. A rule 2. A test 3. A tool restriction 4. A verification step 5. A block 6. A trace that explains what went wrong The article’s most brutal example is even more useful: > 724 seconds. > 100 steps. > Still failed. After the harness was stripped down and redesigned: > 141 seconds. > 19 steps. The model did not suddenly become smarter. The harness stopped making it waste intelligence. And the economics get even sharper: - Jev handles typed decisions at about $0.042 per million input tokens - a thousand gate checks at roughly 200 tokens each can cost under a cent - the same thousand calls to a frontier model can cost around $0.80 before it writes a single word - cached input can cost 20× less than fresh input when the prompt prefix stays stable This is why just use the best model is such a weak strategy. The best model should handle: → hard reasoning → ambiguous evidence → complex code → high-value decisions It should not spend its time deciding whether: - a queue changed - a test needs to run - a file is still untouched - a task deserves escalation - the agent should try the same thing again The frontier model is rented. The harness compounds. Every failure you convert into a rule, test or gate becomes part of your private advantage. The model release will be available to everyone by Friday.
这条是引用着另一条写的：x.com/explosss1ve/status/2108227048024182907