Idea: using Jev as a decision side-car for LLM tool calls
A proposed System One architecture where the LLM only does token prediction while Jev decides when a tool call is needed and injects the results back into the token stream.
This could be a nice architecture to explore: Use @typesafe Jev as a tool call coordinator for LLMs. LLMs are now better at tool calling than they were, but it's still an ugly hack in the coding harnesseses. System One models could likely decide better when a LLM needs to perform a tool call to either get more info or to perform an action - like a LLM's decision side-car of sorts. I imagine a kinda "cyber-LLM" loop that a System One model controls: 1. LLM just does token prediction (let's say 1k at a time) 2. System One model decides if the token stream would benefit from one or more tool calls (issues tool calls, injects input+output into the LLM token stream) 3. go to step 1 🤔