TypeLLM: type-safe generation for LLMs, inspired by Jev
TypeLLM adds type-safe generation to existing autoregressive LLMs on SGLang, supporting JSON Schema, dependency graphs and thinking mode, and is explicitly inspired by Jev.
TypeLLM: LLMs with type-safe generation TypeLLM brings type-safe generation to existing autoregressive LLMs without changing their architecture or weights. Inspired by TypeSafe AI's Jev, it lets models retain their native thinking and free-form generation while producing schema-guaranteed outputs through JSON Schema. Built on SGLang, TypeLLM also supports richer interaction patterns beyond independent typed decisions. Features No out-of-schema hallucinations — Choices stay within the allowed values. Negligible output-token cost — Single-token categorical selection and bounded numeric decoding; optional thinking adds tokens. Shared-prefix reuse — KV caching avoids reprocessing shared context. Dependency-aware execution — Run decisions sequentially, batch independent fields, or declare depends_on to form a dependency graph. Made for open autoregressive LLMs — Use compatible models you already serve with SGLang. Supports thinking mode — Enable reasoning before the final constrained answer. Image input — Pass images to vision-language models alongside the text context. See Image input. Permutation averaging — Reduce option-order bias on explicit enum questions with sampled or exhaustive orderings Github repo - https://github.com/TypeLLM/TypeLLM