Testing Jev's strengths and weaknesses on a $3.40, 24-hour run
The author tests Jev's Choice, Score and Noul decision types, parallel sampling and RLCD confidence mechanism, and concludes the future is Jev plus LLMs rather than Jev replacing them.
I burned $3.40 on Jev in 24 hours. It's not Jev vs current LLMs. It'll be Jev + LLMs. Here are all the goods and bads of Jev. 1. Jev or a Jev-like system will be integrated into all LLMs for automatic model routing, small decisions, if/else queries, retrieval, tool selection, scoring, etc. For all these things, LLMs lose a lot of tokens on thinking and planning and use smarter instead of faster models. That can be fixed with Jev. 2. Jev can do things that require fine-tuning with LLMs. It can learn what to do from just 1 prompt. 3. It's crazy fast. It scans, analyzes, and gives its decisions instantly without a thinking blocker. 4. It doesn't waste tokens on thinking in plain English and planning. It scores probabilities and picks the best one. 5. Output cost of $0. Which makes sense because it's not built for free-form text generation like LLMs. It just outputs decisions. 6. It can fix RAG. The current 32k context is too low, but in the near future, we'll see systems like Jev with a 1M context window, and that'll fix the retrieval problem. 7. Jev has 3 decision types: multiple-choice questions, scoring the probability, and Noul. Choice → Which one? Score → How much? Noul → Yes or no? That's how it doesn't need reasoning. 8. Jev works on parallel sampling. LLMs generate sequentially: Token 1 ↓ Token 2 ↓ Token 3 ↓ Token 4 Jev can evaluate multiple questions against the same state in parallel. That's one of the reasons it has great latency. 9. Instead of RLHF (Reinforcement Learning from Human Feedback), it works on RLCD (Reinforcement Learning for Calibrated Decisions). With RLCD, the model's goal is not just to predict the answer; it also shows how much confidence it has. A model saying: true is different from: true, 99% confidence. 10. Jev response times are roughly: 70–500 ms, which is 193.6× faster and 444.6× cheaper than frontier LLM workflows. The biggest issue with LLMs today is that they take too much time to execute tasks. With a Jev-like system, that'll be at least 50% faster.