Breken catches more production bugs than GPT-6 Astra in 33-product test
In a 33-product evaluation, Breken caught 76 of 235 real production bugs versus 37 for Fable 5.1 and 28 for GPT-6 Astra, with a third of the false alarms.
your tests check what your team thought of, while production checks everything else up until now, real users were basically running that second test. what stood out to me about @brekenai is that it builds a realistic world around your real software, real users, old data, and things breaking on purpose; finds the bugs that only show up in production; fixes them; then keeps checking every release in its evaluation across 33 products, Breken caught 76 of 235 real production bugs, compared with 37 for Fable 5.1 and 28 for GPT-6 Astra, with a third of the false alarms they’re calling it production engineering, the final job in software if you're building a startup, give it a try 👇
This post quotes another post: x.com/brekenai/status/2112936781087973732