Using Jev to judge token cost and risk
The author argues for calling expensive models only when needed, and says Jev can bound a model's scope with probability thresholds so calls are targeted.
AI is the single biggest force amplifier technology ever created for the intelligent generalist. More tk on this, but the next layer of innovation is going to be moving the tech “downstream”. I describe getting max utility out of LLMs as drawing a very tight box with words and blasting the bazooka of training data to totally fill in the space. If you’re not great at a particular form of “math with words” — extremely logic-bounded descriptions of what you’re looking for — the model will leak out and you’ll have to spend a bunch of time fixing things. (In fact, the proper thing to do is usually clear context and attempt the oneshot again.) It takes a particular kind of structural understanding to know the exact scale with which prompting can be applied. The next step involves reducing costs, which means *only* applying expensive tokens when you actually need them. This is sort of a risk judgment, a kind of dynamic “how much is this task actually worth” calculation in tokens as to whether you should be using a frontier model or a cheaper alternative. (FWIW, for everything other than code, I haven’t had to use a frontier model for anything I do, which is usually some form of extreme information processing curated for me to read and digest.) This is where structures like Jev become particularly delightful to use, bc it probabilistically constructs bounds where humans who aren’t quite as good at drawing the box might err. By mathematically constraining where a model is actually applied, you naturally blend efficient math with unwieldy power, enabling precision strikes with a drone rather than scattered aerial bombing. Make no mistake, this is the single greatest time in history to be a sole proprietor or form lean organizations. Why you’re not seeing top-line, large scale improvements should intuitively follow — a scaled bureaucracy, like a megacorporation or a government, simply cannot constrain the output past the low level to make huge leaps. The interaction between AI systems and human coordination requires a level of understanding context that models simply cannot possess. Which is why the biggest cost savings will come from finding needle in a haystack waste and redundancies, the “spaghetti code” where money has been used to patch inefficiencies in org structure. (This doesn’t seem like a big deal, bc companies don’t trade off of revenue multiples anymore, but it reduces implicit leverage, and definitely makes outsourcing etc. more efficient.)
This post quotes another post: x.com/nateafischer/status/2102957907302105113