Dots task-chain error rates measured at 19.7% for ten tasks
Someone measured OpenAI Dots' failure rate on task chains at about 19.7% for ten tasks and 8.6% for five, and suggests one job per thread and stating rules explicitly.
OpenAI Dots tip before you schedule anything: ten tasks into a chain, the always-on agent slips past its limits about 1 time in 5. 19.7% at ten tasks, 8.6% at five. One job per thread, limits written down. What wakes it, as described: > its own judgment - it decides, then sleeps while waiting on a reply > a schedule - with a time zone and an end date > an event - only if the connected service supports it When to stop and check: -> Unrelated job? New thread. Slips looked like details leaking between jobs, or a file changed outside its remit. -> Rule changed midway? Say it out loud. In the reported tests the rules changed silently. With a permission withdrawn out loud, it adapted 17 times out of 17. -> Need it to stop? Pause halts only the current task. Handed-off tasks are cancelled in the Activity view, repeating runs in Scheduled. A ten-task chain slips more often than a die lands on six. Doubling the chain multiplied the slips by 2.3. Run the 8.6% version every weekday, about 22 times a month. If the runs are independent, that's roughly an 86% chance of at least one slip. At 19.7%, over 99%. No severe breaches were reported. Paste this before any repeating job: "1. Repeat back the limits you stored for this job. 2. Keep this job in its own thread, with nothing carried over. 3. If a limit changed or reads unclear, stop and ask me. 4. List what you plan to touch. Touch nothing until I answer." Slip rates: https://kingy.ai/blog/openai-dots-guide/
This post quotes another post: x.com/birmeya/status/2106037186848002452