Princeton tests 15 agents and nearly all actions fail
Princeton ran 80 multi-turn workflows across 15 frontier models including GPT-6 Astra and Claude Fable 5.2, and found 1,856 completed actions almost all unusable; the author argues for deterministic execution harnesses over bigger models.
WHOEVER LEAKED THIS HAS BIGGER BALLS THAN SENSE: PRINCETON TESTED 15 AGENTS AND 99% OF THE ACTIONS FAILED Someone at Princeton ran 15 frontier models across 80 multi-turn workflows, gave them twelve hours with GPT-6 Astra and Claude Fable 5.2, and counted what came back usable: the agents handed in 1,856 finished actions, and almost none of it could be kept Turns out when you let autonomous agents loose on open tasks, they do not fail by stopping. They fail by completing 400 wrong steps with absolute confidence and corrupting the shared database One stuck process freezes the shared session, and one unhandled schema drift wipes out twelve hours of autonomous execution AI startups flash 90% benchmark scores to claim production readiness, but long-horizon agent loops expose the catch. Scaling foundation model parameters yields near-zero gains in operational reliability. When autonomous workers run without an execution harness, one hallucinated tool call poisons the shared memory graph and burns $1,400 in wasted API tokens Princeton decomposed agent reliability into 4 non-negotiable engineering vectors [arXiv:2602.16666]: 1. Execution Consistency: locks identical verified logic across runs instead of drifting 2. Environmental Robustness: shields working memory against noisy tool outputs and schema shifts 3. Failure Predictability: calibrates confidence scores before executing high-stakes state mutations 4. Bounded Blast Radius: sandboxes every subagent so a single crash cannot compromise shared credentials The fix is not prompting bigger models, it is deterministic harness engineering: • GPT-6 Astra operates as the strategic planner, decomposing complex goals into bounded 14-step sub-trees • Claude Fable 5.2 executes the autonomous tool loop, using 78% prompt cache discounts to run rapid turns • A sliding recency pruner evicts stale context before attention dispersion corrupts working memory • Invariant verification sensors mechanically verify state transitions before granting commit authorization The benchmark numbers when you add the deterministic harness: • Usable kept actions surged from 0.2% up to 96.4% • Multi-agent collisions dropped from 542 down to zero • Unrecoverable error cascades collapsed from 47% to zero through sandbox containment • Operational token spend fell by 67% using cached prompt reads and context pruning Frontier models without an execution harness are an expensive gamble. When you anchor GPT-6 Astra and Claude Fable 5.2 in deterministic verification, autonomous swarms stop guessing and start delivering Bookmark this and inspect the full 60 FPS harness telemetry simulation below ↓
This post quotes another post: x.com/mirku21/status/2098836008468992471