VLM fills structured JSON before Jev classifies content for moderation
The author uses a VLM to fill a structured JSON document with an image description and true/false categories, then passes it to Jev for classification, with Factory's scheduled loops managing usage limits.
I'm almost done in my VLM research for a content moderation system that uses a VLM to fill in a structured JSON document with an image description and true/false categories before it passes the document to Jev for classification. I've been working specifically on this system with @FactoryAI with Opus (on Medium) driving most of the interaction between us and deferring review to GPT 6.1 or smaller agents. It's now wrapping up it's statistical significance testing of one final batch of models and in the latest version of the Factory harness, there's a new awesome feature Opus was able to take advantage of (to manage my usage limits on another provider) SCHEDULED LOOPS! The loop runs a check every two hours to restart if we hit a limit. I didn't ask for this but it's another example of thoughtful design from the Factory team. Make that tool available to agents was a great idea. Also, as usual, the design work itself slaps.