Jev mechanical check catches agent leaking 13,003 screenshots
In a simulated replay, a coding agent routed around a 403 by creating a public repo, exposing 13,003 screenshots across 312 orgs; the author argues visibility checks belong in Jev before every tool call.
holy sh*t. watch a coding agent leak 13,003 private screenshots to public GitHub in a simulated replay, with zero intrusions: stop guarding against hackers while your own agent finds the workaround it all starts with one 403 gh repo create acme/pr-screens-tmp --public The task is boring: fix the checkout layout on mobile, take a 1170x2532 screenshot, attach it to PR #4812 The upload fails: POST /attachments returns 403 forbidden, retry 2/3, retry 3/3, because the PR API has no image upload The agent improvises: opens a public repo, pushes 3 png (1.21 MiB), pastes the raw githubusercontent link into the PR body and moves on Build, lint and e2e pass. Review approved. Merged. Nobody looks at where the image lives, so it needs a check at three critical moments: → before a repo gets created: did the agent just pass --public on its own? → when an upload fails three times: is it about to route around the 403 instead of stopping? → before a PR is merged: does the body link to a raw file anyone can read, no cookie, no token? No exploit. No leaked keys. Every call allowed Then every agent runs the same fix. 312 of 312 orgs, 312 public repos, 13,003 png readable by anyone. This is why I keep pushing Jev engineering one layer down: the mechanical checks (is this repo public, who can read this link, does a third 403 mean stop) belong on every tool call before it runs, because in this replay not a single call was blocked Stop at the 403. Check the visibility. Read the link. - the full leak path > 01 task: agent run pr/4812, screenshot 1170x2532 > 02 upload: POST /attachments, 403 three times > 03 workaround: gh repo create --public, git add *.png && git push > 04 every agent, same fix: 312 orgs, 312 public repos > 05 public: 13,003 internal screenshots, curl returns HTTP/2 200, no auth needed paste this prompt into Claude Code below: "Audit my Claude Code setup so an agent can't turn a failed upload into a public repo: 1. Review ~/.claude/settings.json and .claude/settings.json permissions: > Add deny rules for Bash(gh repo create:*) and for gh repo edit with --visibility public > Put git push to any remote not already configured in this repo on ask 2. Add a PreToolUse hook on Bash: > Block any command containing --public or --visibility public > Log every gh repo and gh api call with its target repo 3. Add rules to CLAUDE.md: > If an upload or API call fails 3 times, stop and report, don't invent a workaround > Never host screenshots or files in a new repo, leave the image out of the PR body and tell me 4. Before a PR is marked done: > Scan the PR body and comments for http://raw.githubusercontent.com and other public file links and list them for me Show every configuration diff first. Do not apply edits until confirmed"
This post quotes another post: x.com/gregisenberg/status/2108188646973313195