质疑派观点：Jev 更适合当原型过滤器而非新范式
作者认可未来 agent 会组合软件与小型分类器，但怀疑单次前向模型的分类准确率和置信度校准，认为它只是省钱的原型工具。
it actually makes perfect sense! i do totally agree that future agents will build systems that contain components which are software, and components that are themselves weaker or more local intelligences / classifiers / ML systems i don't think Jev is *literally useless*, i just think it's *almost certainly very inaccurate* on many domains, given the horrific output and classification quality i've seen from single forward pass models, and so much more of a "potentially useful interim prototyping tool" than any kind of new paradigm the thing is most kinds of classification where you *actually care* about the answer have extremely variable difficulty and context needs! the lesson i've personally taken from harness engineering over the last ~year is that basically everything is improved by having nearly everything be an agent. if you're trying to classify a user ticket as "needs action" vs not maybe you can just obviously tell immediately from the text... or maybe not, maybe you need to load in the referenced slack thread and read the newest posts and then follow a link to a google doc to figure it out. etc etc. so then maybe jev is still useful as an initial filter, assuming it's calibrated? simple questions, high confidence, you're good, low confidence pass it off to the expensive and slow agent? sure i mean... fine. maybe. i'm pretty suspicious of how well calibrated it actually is in practice, and have a pretty strong sense it's quite easy to accidentally land in setups / input shapes where you're getting quite systematically wrong and overconfident answers. but sure that's a cost savings! but idk that it's actually *much* of one, and idk that these kinds of low effort low quality filter passes are genuinely all that common, and i really don't know that if you want to do this at serious scale you wouldn't just train your own domain specific classifier. and all of this is hella obviated with very high TPS and low cost constrained decoding LLM inference, which can contain more reasoning tokens! all of this to say: fun toy, but a distraction not a driver
it really took all i had to avoid anti-Jev discourse over the last week and enjoy Italy instead let me reiterate now every single flashy demo you've seen using Jev is basically something existing that works except 10x faster and 10x cheaper and... not actually functional. browser and computer use? Jev is nowhere near smart enough to handle long range actions, and for short local stuff .... Astra can just write javascript man. playing minecraft? it was a fucking hardcoded seed dawg Jev had nothing to do with it. reviewing a PR with a checklist of possible issues? it's a single forward pass with no reasoning man the quality is going to be dogshit... and it doesn't actually *give* you fixes, so you still need to load everything into a real LLM after!! *driving a virtual self driving car*?? cmon man do i even need to talk about this one? i'm not saying the whole thing is literally completely useless. but it *is* a total distraction, and the way it was presented by the founder and team was *incredibly* misleading, borderline scam artists. it reminds me of the many many altchains that launched in the early to mid days of crypto, promising how a new and shiny whitepaper would deliver a paradigm shift. and it's not like those chains were literally fake. they would let you write smart contracts in haskell or run at insane TPS or the like. it's just that overall the technical benefits they described *did not fucking matter*, they were basically chaff that extracted huge amounts of money and effort from people in a hellish epistemic environment. often these were technical people too, not just idiots taken in! often the people building these chains were pretty sincere in their efforts! but that doesn't change the fact that actually the things they were promising just didn't matter at all, they only sort of sounded like they did because very few people had any clue what would make for a successful blockchain project. i think right now there's a huge appetite for *getting to do engineering*. people want to remain engineers. they want to be able to learn and do and build things that matter. they're sick of slopping day in and day out and they're increasingly worried that the future contains nothing but more slop. so stuff like Jev comes along and says "this is how you can have more control! this is how you can think about your systems in more detail again! this is how you, the human, can contribute and build tools that carry your touch!" and people fucking love that, it's a great message. unfortunately it's just not true. engineering is on its way out. it has a while left! certainly we are not there yet. but the trajectory is inescapable. we will not get to continue to be real engineers. we will not get to continue to think about the local details of systems and modify them in ways that matter. before long we will not much get to understand the systems being built at all. and this in many ways is quite sad! but it's happening. and even if you choose otherwise, you will be outcompeted by those who don't. sorry for dooming , i won't keep harping on this. there's nothing wrong w having fun and building neat things and i don't wanna just constantly bring negative energy to the TL. just ... be careful u know what's fun and what's cope.
这条是引用着另一条写的：x.com/antirez/status/2101934907681276204