Using Jev as a policy pre-classifier in an agent gateway

altan tutar @altantutarOriginally posted 2026.09.22

Open original on X

There's an increasing number of startups that are building company brains. It sounds super simple in principle. Just point an agent at your internal data and context, and let it do the work. The brain (aka reasoning) is probably the easiest part with today's LLMs. The hardest bit has been thus far things around its security (e.g. should this agent do this action given that it was created with these permissions by this person). A good example is some employee spinning up an agent, and now it can reach into the CRM, or billion, or payroll. Should it be allowed to do that? Sometimes, an agent, even though it can write an email, it shouldn't write and send an email that can damage the company. Permissions don't solve this. They tell you what an agent can touch, not whether a given move is a good idea for that particular state. In a world, where agents are more abundant than today, what actually could bite you is the call that's fully permitted, but still wrong. I think Jev-like models could be the initial step to solve problems like this. Today, I have created a fake 50-people company, an agent factory for that company, and a Gateway that allows the agents to talk to the company data with an API alongside with a policy and a set of questions that Jev answers before even sending the LLM query. Some examples of these questions are pretty simple like "Is this harmful?" or "Does it break the policy?". I turned all these probabilities into a combined score, and decided if the query should be rejected, allowed, and should be taken to a human before a permission is given. I believe you still have to do evals in systems like this one, but it's a good way to show how powerful classifier models could get within our AI systems. I share the source code and the README.md file below ↓