Constraining an ops agent's action space with Jev
The post proposes "constrained creativity": keeping an ops agent inside a DSL and using Jev to return typed choices and scores, cutting root-cause analysis cost by up to 12x without losing accuracy.
Introducing Constrained creativity. TL;DR: Constraining an ops agent's action space makes it explainable and cost-effective, without hurting accuracy. AI agents hold promise to automate systems operations. However, it's the engineers who stay accountable for every change, and they need explainability to trust the agent. By and large, an ops agent needs to come up with pipelines that are: • Effective: right answers, even on incidents it has not seen • Efficient: keeps pace with alert volume, at a reasonable cost • Explainable: an evidence trail an engineer can check An LLM writing free-form code might reach the right answer, but its work is hard to check and its exploration and thinking keep burning tokens. Our pitch: constrained creativity. Give the agent only as much freedom as the task needs, and let it be creative inside those limits. More room than a fixed playbook. Far less than arbitrary code. But doesn't this sound backwards? Shouldn't an agent facing new cases get unlimited freedom? History has plenty of counterexamples. Jazz improvisers create inside fixed chord changes. Dr. Seuss wrote Green Eggs and Ham on a bet to use only 50 words. So much so that a 2017 psychology study found the same pattern and named it the Green Eggs and Ham hypothesis: constraints help creativity. Recent agent systems already apply this idea implicitly: • Jev (TypeSafe) and OpenAI's Decisions API: the model returns a typed choice, score, or yes/no • CaMeL (Google DeepMind): the agent writes restricted Python that a policy checker runs • Grammar-constrained decoding: output must parse On similar lines, we propose giving ops agents a domain-specific language (DSL) and letting them be creative inside it. The results? Up to 12x cheaper root cause analysis without losing accuracy, and red-teaming agents that finish multistage attacks free-form LLMs could not. With @vipulharsh93 @LesleyChou339 @MarkoMorrisonX @badityap @vyas_sekar @HuiConviva001 👇 Paper, blog, and how it works for RCA in the replies