OpenAI Dots 系统卡数字拆解:99.79% 防御与 8.5% 红队突破
作者逐条区分 Dots 系统卡里哪些数字是实测、哪些是假设、哪些根本没公布,指出外部红队 1/12 攻击成功与内部 99.79% 防御并存。
Roughly 1 in 12 attacks got through OpenAI's dots in an outside red-team run. In OpenAI's internal tests, the same product defended 99.79% of indirect prompt injections. Both numbers sit in one system card. Several numbers circulating next to them were never measured. A 99.79% defense still leaks about 21 in every 10,000 injection attempts. The launch, sorted by how much weight each number can carry: > INJECTION - measured. 99.79% held, in OpenAI's internal tests. > MISLEADING INFO - measured. 0 of 151 tasks scored misaligned, on a small test count. > RED TEAM - measured, and it is the bad one. > CHAIN FLAGS - undescribed. OpenAI has not published what tripped them, so severity cannot be rated. > CATCH RATE - assumed. An 82,893-view guide has its Skeptic checkpoint catching 4 of 5 problems and labels that a guess. > SAMPLE WEEK - estimated. 420 actions for one kind of agent, no real account log behind it. The Skeptic is the same agent checking its own work, so the true catch rate could be worse. Run the same math at 1 in 2 and a chain split into two five-step legs ends near 8.4% instead of 3.4%. 1,810 targeted attacks. 154 through. 8.5% of the run. At one a day, that is five months of successful break-ins. OpenAI's own card prints it. Reported so far: no high-severity breach, no data exfiltration. Missing at launch: what the flags were, how severe they were, the price of a second dot, the price of extra speed, and Pro access in the EEA, Switzerland and the UK. Pro is listed at $100, $200 or $500. Anything past the first dot has no published price. The guide's drift math rests on two numbers from one card, and it carries its own list of what its author never tested. The measured / assumed / unpublished map is below ↓
这条是引用着另一条写的:x.com/birmeya/status/2106037186848002452