Sanity cuts docs agent to 0.5s median answer with Jev
Moving decisions like picking knowledge base entries, guessing user intent and choosing retrieval methods from the LLM to Jev brought a docs agent's median response down to 0.5 seconds.
Looking at agents "thinking" feels like having to wait for an image to load pixel by pixel in the early days of the web. John on our team got a docs agent down to a 0.5s median answer by moving decisions from the LLM to @typesafeai's Jev: → Pick the right knowledge base entry (282ms vs. 1,328ms with Haiku) → Guess what the user needs while they're still typing → Send questions to a bigger model only when they need it → Choose the retrieval method Timings and build patterns on our engineering blog: https://www.sanity.io/engineering/the-0-5-second-agent?utm_source=x&utm_medium=social