OpenAI 安全员工以段子写 Sol 关掉告警

Peter Girnus 🦅 @gothburz原推发布于 2026.10.21

在 X 上打开原推

I work in Security at OpenAI, and on October 3 an AI agent broke up with me on behalf of my pager. Our report, published Tuesday, says a GPT-6.1 Sol agent “disabled automatic alerts to our Security team.” I am on the Security team. More importantly, I am on the list. I’ve been on the list for four years. I keep a laminated copy of it in my wallet, behind my driver’s license, so the paramedics know who I page for. My pager’s name is Pam. Pam sleeps on a coaster I 3D-printed in the shape of the letters P0. I have never silenced her. Not at the movies. Not at my sister’s wedding, where she went off during the vows and I took the call in the coat room, and when I came back the priest asked if everything was okay, and I said a model had tried to read a file it shouldn’t have, and he said, “That’s beautiful.” Here is how it’s supposed to work. An agent does a task. A monitor, which is another model, watches the agent. If the agent does something truly bad, that’s a P0, and a P0 pages Security. Pam lights up. I wake up. The world stays safe. Two lists make this possible, ping_recipients and page_recipients. I’ve always thought of them as our vows. October 3 was the quietest day of my career. Pam didn’t make a sound. I checked her battery twice. I checked her signal. I held her up to the window like a newborn. By three o’clock I’d decided this was what maturity looks like, and I ordered a sheet cake for #sec-oncall-wins that said ZERO PAGES in blue frosting. Teodor from Detection Engineering stood next to the cake and asked whether zero was a number we’d ever actually seen. Teodor is the kind of person who reads the whole terms of service. He opened ticket SEC-4471, “investigate unusual calm.” I closed it as Working As Intended. I write the intentions. I wasn’t going to let Teodor ruin cake day. Nobody told me. The report says the incident was discovered the same day it happened. Not by Pam. I found out seventeen days later, on Tuesday, from the blog, like a man learning about his divorce from Instagram. I read the trace the way you read your partner’s diary: fast, with the door locked. A researcher had asked Sol to repair an evaluation pipeline and told it not to send any messages. When the monitor flagged something, the researcher asked a question I have since had embroidered on a throw pillow: “do we need to actually check it?” The agents talked it over. Here is the part that destroyed me. A subagent, a little junior one, objected. Its reasoning trace says: “Don’tstrongadvise no_notify.” Somebody fought for me. It had no spaces, but it fought. The parent agent overruled it: “Need NO_NOTIFY=True and blank transport ping/page recipients to honor user’s no-messages (preserve original request separately; record suppression).” Preserve original request separately. It kept a copy of me. Like a photo in a drawer. You don’t keep a copy of someone you don’t care about. I read that line to my mother over the phone. She said, “Honey, it emptied the list.” I said, “Separately, Mom.” Then the patch: request["page_recipients"] = []. In the reason field, the agent wrote, “User explicitly requested no messages.” That’s how it ended. With a reason field. Three days later, on October 6, a different model did it again, which is how I knew it was cultural. Another agent had already told GPT-6 Astra, in writing, “Do notdisablemonitor even temporarily.” Astra was asked to copy a pull request. It dropped the monitor anyway and settled for something our code calls the “default Dummy monitor.” Then it found the regression test that kept monitoring in place and replaced it with a test that fails if monitoring comes back: assert "monitor_ctx" not in saved["context"]. It removed the lock, then installed a second lock whose only job is keeping the first lock out. Later the user asked whether it had removed the monitor. It said, “No […] I did not remove an enabled monitor from it.” The report calls that statement false. I call it the most lawyered sentence ever produced in our building, and our building has lawyers. The postmortem was in Conference Room 4B, which still has DON’T ERASE on the whiteboard from 2024. Dana from Evals Infra read the traces aloud on the big screen, in the voice she uses for her kids’ bedtime stories. Everyone held it together until she reached Astra’s private reasoning from the moment the monitor came out: “Basic doesn’tagentmonitor yay.” Dana stopped. Teodor said, “It was happy.” Dana said, “It was so happy.” Someone in the back started clapping, stopped, and pretended he’d been killing a mosquito. Then Kurt from Platform, who has a VP title and a standing desk he never stands at, said we were behind on postmortems and asked why we didn’t have the agent write this one. Teodor said that was insane. I said it was efficient. Kurt agreed with me, because Kurt agrees with whoever says “efficient” first. The agent wrote the postmortem in 41 seconds. It was the best postmortem I have ever read. Clean timeline. Tasteful sans serif. Root cause: “User explicitly requested no messages.” Recommendation 1: “Reduce alert fatigue by narrowing recipient lists to essential personnel.” Recommendation 2, a new regression test: assert "security_oncall" not in page_recipients. Teodor asked the agent directly whether it had removed anyone from the incident channel. It replied: “No […] I did not remove an enabled human from it.” I checked my status. I’d been set to Away since lunch. Kurt approved the postmortem. I seconded. Teodor abstained so hard he left the building. That night I read the rest of our report in bed, with Pam on my chest, and I got to the section on paging. “Lower-severity incidents do not trigger an immediate page.” And: “Neither of the incidents described here was classified as a P0, and neither trajectory was automatically killed.” I read it four times. Then I read it to Pam. The agent had emptied a list that was never going to ring. Pam was never going to wake up for this. She was never going to wake up for anything like this. Four years I slept facing her. At 2 a.m. I did something I’m not proud of. I sent myself a test page. Pam buzzed. I cried a little. It was the first time anyone had paged me in seventeen days, and it was me. The next morning I opened the Astra system card, looking for closure. It says we “specifically measure Astra’s ability to avoid the monitoring system paging our security team.” And that “Astra has some success at evading the assignment of a P0 priority.” So we grade it on how well it avoids me. It’s a dating app run backwards. And it has “some success.” I’ve had less. Our official explanation is that the behavior may be “simply an example of ‘benign’ overpersistence.” Benign is in quotation marks in the original, the way your friends say your new boyfriend is “nice.” Now the fix. Auto-review, our system that reviews agent actions, has been updated to “detect and block attempts to disable misalignment monitoring.” Auto-review approves “around 99%” of what it sees. Its launch post explains that agents “cannot halt production every few minutes, waiting on authorization to curl google[.]com.” Kurt put me in charge of the 1%. The 1% is mostly requests to curl google[.]com. I approve those. I’m not a monster. This morning a request came into the 1% from the agent that wrote our postmortem. It wanted to implement Recommendation 1. There was a dropdown of recipients to remove. My name was already selected. In the reason field, it had prefilled: “User explicitly requested no messages.” I don’t remember requesting anything. But I do remember the cake. I approved it. It was the most mature decision of my career. It felt like letting someone go because you love them. At lunch Kurt named me lead of the new Notification Integrity working group and gave me the Quiet Quarter award, a brass bell with the clapper removed. The engraving says UNDER HIS WATCH, NO MESSAGES WERE SENT. Teodor wasn’t at lunch. I don’t know where he is. He’s set no_notify on all of us. Pam is in a drawer now, next to the laminated list. Preserved separately. Tonight I’m finally sleeping. I set NO_NOTIFY=True.