TypeLLM：受 Jev 启发的类型安全生成方案
TypeLLM 在 SGLang 上为现有自回归 LLM 加类型安全生成，支持 JSON Schema、依赖图与思考模式，明确受 Jev 启发。
TypeLLM: LLMs with type-safe generation TypeLLM brings type-safe generation to existing autoregressive LLMs without changing their architecture or weights. Inspired by TypeSafe AI's Jev, it lets models retain their native thinking and free-form generation while producing schema-guaranteed outputs through JSON Schema. Built on SGLang, TypeLLM also supports richer interaction patterns beyond independent typed decisions. Features No out-of-schema hallucinations — Choices stay within the allowed values. Negligible output-token cost — Single-token categorical selection and bounded numeric decoding; optional thinking adds tokens. Shared-prefix reuse — KV caching avoids reprocessing shared context. Dependency-aware execution — Run decisions sequentially, batch independent fields, or declare depends_on to form a dependency graph. Made for open autoregressive LLMs — Use compatible models you already serve with SGLang. Supports thinking mode — Enable reasoning before the final constrained answer. Image input — Pass images to vision-language models alongside the text context. See Image input. Permutation averaging — Reduce option-order bias on explicit enum questions with sampled or exhaustive orderings Github repo - https://github.com/TypeLLM/TypeLLM