Astra blog hallucination rate changed to 2%, then reverted to 4.2%
Internet Archive snapshots show Astra's hallucination rate was changed from 4.2% to 2% on launch day, and GPT-5.6 Sol's from 12.2% to 9.4%, before both were changed back.
Among the most notable changes was Astra’s reported hallucination rate. In the first internet archive snapshot of the blog post from 2:23 p.m. ET, it was 4.2%. It remained that number for several more snapshots, the last being a fifth at 3:11 p.m. ET—about 10 minutes before OpenAI tweeted out the final version. But the hallucination rate, along with four other metrics, changed in the sixth archival snapshotof the page taken at 5:20 p.m.—after everyone could likely finally see the blog. It was halved down to 2% for Astra. The scores for Astra’s predecessor, GPT-5.6 Sol, also went down from 12.2% to 9.4%. OpenAI has continued to change this metric; as of this writing, the hallucination rates are back up to their original 4.2% and 12.2%.
This post quotes another post: x.com/jeremyakahn/status/2096029945856242085