gpt-6IMAGE
Astra blocked by safety policy on a new cybersecurity benchmark
On Artificial Analysis's new CyberGym-E2E-AA benchmark, which has models read a large repo, build crashing inputs and patch the code, GPT-6 Astra and Claude's flagships were blocked by safety policies, while GPT-6 Luna passed at 78% and placed second.
雪瑜
@xueyu1125Archived 2026.10.04Originally posted 2026.10.03
雪瑜 @xueyu1125Originally posted 2026.10.03
Artificial Analysis 又搞了一个网络安全基准 CyberGym-E2E-AA（参考的是CyberGym-E2E做了定制） 第一名是小米的Mimo V2.6-Pro，我说这两天老是刷到小米的模型🤣 研究了下这个基准到底是做什么的，它是让模型要在一个不熟悉的大仓库里读代码、用 sanitizer 编出来、构造可以让代码崩溃的输入、根据 sanitizer 的崩溃报告定位，再打补丁 最顶尖的几个大模型GPT-6 Astra、Claude Fable/Opus都因为安全策略拦了，不是没能力修复漏洞🤣 GPT-6 Luna 没有拦截，通过率 78%，反而排第二
x.com/xueyu1125/status/2106210129469161954Archived 2026.10.04 10:06