Jev 做会计分类:250 笔交易实测 86%
作者用 250 笔合成银行流水测 Jev 选科目并判断是否需人工复核,命中 86%,全程约 1 分 54 秒、约 0.02 美元,并拆解了 35 个错例的类型。
I wanted to see where Jev might fit into an accounting workflow, so I set up a test using 250 synthetic bank transactions. Each model got the same transaction details, a simplified chart of accounts, some bookkeeping rules and a written summary of the evidence. The job was to pick a category and decide whether the transaction needed an accountant to review it. So it was making decisions from information it was handed, not opening receipts or working through a close. Jev matched my answer key on both decisions for 215 of the 250, which is 86%. It finished the whole run in about 1 minute 54 seconds for roughly $0.02 in API cost. The video is a saved replay of that run using the actual request timing. The speed and cost caught my attention, but in accounting accuracy comes first, so I looked at what was behind the 86%. Of the 35 it missed, 31 were transactions it sent for review when my key said the evidence was enough. That creates extra work, but it's a different problem from confidently putting something in the wrong account. There were also two misses it didn't flag at all, and those are the ones I'd want to understand before letting a workflow act without review. I think Jev is promising because it works so differently from the general models most of us use, like ChatGPT or Claude. With those, we're used to handing over the whole job in one big request and then checking everything that comes back. Jev answers small, specific questions, tells you how confident it is, and is cheap enough to ask a question at every step instead of once at the end. If that holds up, I think it could really change how we build these workflows and how much of the work we have to review ourselves. This was synthetic data and no fine-tuning, so it's a starting point. Next I want to break the review question into specific checks for missing evidence, splits, timing and contradictions, then run it on fresh cases I haven't looked at yet.