Tag: #llguidance
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Structured Outputs: The Cost of Masking "Illegal Characters" Per Token
Just before the model selects the next token, structured outputs mask the probability of any grammatically invalid tokens to zero. As a result, even a 0.6B model with almost no conversational ability produced valid JSON
2026-10-27 · 17 min read #vLLM#구조화 출력#structured outputs#JSON 스키마#xgrammar