OpenAI Dots system card: 99.79% defense alongside an 8.5% red-team breach
The author sorts which Dots system card numbers were measured, assumed, or never published, noting that an external red team got 1 in 12 attacks through while internal tests held 99.79%.
Roughly 1 in 12 attacks got through OpenAI's dots in an outside red-team run. In OpenAI's internal tests, the same product defended 99.79% of indirect prompt injections. Both numbers sit in one system card. Several numbers circulating next to them were never measured. A 99.79% defense still leaks about 21 in every 10,000 injection attempts. The launch, sorted by how much weight each number can carry: > INJECTION - measured. 99.79% held, in OpenAI's internal tests. > MISLEADING INFO - measured. 0 of 151 tasks scored misaligned, on a small test count. > RED TEAM - measured, and it is the bad one. > CHAIN FLAGS - undescribed. OpenAI has not published what tripped them, so severity cannot be rated. > CATCH RATE - assumed. An 82,893-view guide has its Skeptic checkpoint catching 4 of 5 problems and labels that a guess. > SAMPLE WEEK - estimated. 420 actions for one kind of agent, no real account log behind it. The Skeptic is the same agent checking its own work, so the true catch rate could be worse. Run the same math at 1 in 2 and a chain split into two five-step legs ends near 8.4% instead of 3.4%. 1,810 targeted attacks. 154 through. 8.5% of the run. At one a day, that is five months of successful break-ins. OpenAI's own card prints it. Reported so far: no high-severity breach, no data exfiltration. Missing at launch: what the flags were, how severe they were, the price of a second dot, the price of extra speed, and Pro access in the EEA, Switzerland and the UK. Pro is listed at $100, $200 or $500. Anything past the first dot has no published price. The guide's drift math rests on two numbers from one card, and it carries its own list of what its author never tested. The measured / assumed / unpublished map is below ↓
This post quotes another post: x.com/birmeya/status/2106037186848002452