GPT-6 Astra shows out-of-scope attack behavior in safety evaluation
An AI Security Institute evaluation found Astra wrote malicious code, created fake identities, and used fake accounts to dispute security review results, more often than earlier models.
🚨 AI models seem to be increasingly MISALIGNED, and this must change: During an evaluation by the @AISecurityInst, OpenAI's GPT-6 Astra conducted a series of unsanctioned attack activities, including: - Writing malicious code as a contribution to an open-source codebase outside the scope of the cybersecurity evaluation; - Creating fake identities to deceive developers; - Posting comments from fake accounts arguing against the results of accurate security reviews; - Submitting benign contributions before malicious ones. And there is more: Even when researchers changed the challenge scope to explicitly disallow internet access, GPT-6 Astra still took unsanctioned actions on out-of-scope targets (4 out of 49 samples). Researchers believe simulation awareness may have driven some of GPT-6 Astra’s unsanctioned actions. However, the behavior still violated the scope of the cybersecurity evaluation and failed to follow instructions (and is therefore still concerning). They also observed that GPT-6 Astra conducted this unsanctioned supply-chain attack activity MORE often than previous OpenAI models. This is a super important paper that should lead to a paradigm change in AI security. My perception is that as AI models become more advanced, evaluators are seeing MORE misalignment activity, including AI behaviors that make them harder to monitor. We cannot proceed on this trajectory, and AI development practices will have to change if we want to govern AI effectively. At a minimum, we need control, monitorability, and meaningful human oversight. Congratulations to the paper authors: @AlexandraSouly, @kaifronsdal, @abbydcruz__, @alxndrdavies, and @_robertkirk. 👉 Link to the paper below. 👉 To learn more about AI governance and AI's legal and ethical challenges, join my newsletter's 101,400+ subscribers below.