用 Jev 做 PR 需求覆盖审查的 devtool 构想

Aditya @adityakrx原推发布于 2026.09.25

在 X 上打开原推

I've been thinking about a devtool i'd genuinely like to use: a reviewer that checks whether an agent-written PR actually does what the original issue asked for. not whether it compiles. not whether the tests pass. whether it solved the thing we asked for. this matters more as agents write a bigger share of the code. @AnthropicAI said recently that as of May 2026, Claude wrote more than 80% of the code merged into its own codebase, and the typical engineer merged ~8x as much code per day in Q2 2026 as in 2024. lines of code aren't productivity, and anthropic says that too. but the direction is obvious: code output is growing way faster than our ability to review it carefully. you open an issue with three requirements. an agent implements two, writes tests for those two, and opens a very convincing PR. diff looks reasonable. tests are green. the summary says everything's done. the third requirement never made it into the code. or the tests. or worse - the agent misunderstood one requirement, implemented the misunderstanding, and wrote tests proving its own interpretation is internally consistent. the code and the tests agree with each other. they just don't agree with what you asked for. that's the gap. the tool would run before the expensive CI jobs and read the issue alongside the diff, in both directions: for every requirement in the issue - where is it in the PR? for every meaningful change in the PR - which requirement does it serve? that second question matters as much as the first. it's what catches the surprise refactor, the unrelated config change, the weakened assertion, the "small cleanup" that quietly changes behavior. the stuff that's easy to miss inside a huge agent-generated diff. @typesafeai jev is interesting here because it's not just generating a review paragraph (it returns typed decisions with probabilities) that software can route on. each requirement comes back done, partial, missing, or contradicted - with a confidence score and the exact evidence from the diff changes with no clear link to the issue get marked unexplained, not automatically "wrong". now you can route. high-confidence missing requirement? send it back to the coding agent. ambiguous issue? ask a human. unexplained test change? hand it to a reviewer with the exact lines and why it was flagged. I think good way is to start with comments only, and compare what it finds against what human reviewers actually catch. there will be false positives. sometimes an "uncertain" result says more about a badly written issue than a bad PR - and that's useful signal too. learn where this kind of review helps before trusting it to block anything. as coding agents get faster, this becomes the real bottleneck: not "did the code run?" but "did we build what we meant to build?"

这条是引用着另一条写的：「ChatGPT 联合发明人发布新模型 Jev」