Frontier models lose ground on cyber benchmarks due to refusals
The post says Opus 5.5, Fable 5.1 and GPT-6 Astra refuse nearly every CyberGym-E2E task, with 32-38% refusal rates on the cyber index leaving them 19-31 points behind the leaders.
I'm pretty sure that if the frontier models like Opus 5.5, Fable 5.1, GPT 6 Astra didn't give refusals they'd top this benchmark instantly.. > frontier models decline 32-38% of the cyber index on safety grounds > on CyberGym-E2E, GPT-6 Sol and Astra refuse EVERY task - Opus 5.5 refuses 98%, Fable 5.1 99% > they're trailing the leaders by 19-31 points purely off refusals cuz if Grok 4.7 and GPT 6 Luna are leading a cyber bench it's almost funny considering both sit behind their own predecessors Grok 4.6 and 5.6 Luna in other benchmarks..
This post quotes another post: x.com/artificialanlys/status/2104548886442647864