Top stories

Ubicloud enabled constrained decoding in vLLM to enforce valid tool-call schemas for Kimi and DeepSeek models, reporting an increase in K2VV ToolCall benchmark schema accuracy from 92% to 100%. The company had observed that Kimi K2.6 served through vLLM 0.28 produced the expected tool-call schema 92% of the time, meaning roughly two of every 25 calls were malformed. Invalid calls can add error messages to an agent’s context, reduce output quality, and require retries that increase costs. Tool calls require an LLM to emit structured arguments matching a declared JSON schema, so an agent harness can invoke functions such as web-page fetching. Constrained decoding prevents the inference engine from selecting tokens that would violate the required tool-call structure or JSON schema during generation. vLLM uses structural tags from XGrammar to restrict valid tokens at each stage, including tool names, call indexes, and arguments. Although vLLM added constrained decoding in version 0.25, it does not enable strict tool decoding by default for all requests because support is model-specific and can cause problems for some models. vLLM later added the server-side --tool-strict-level flag in version 0.31. Moonshot AI recommends constrained decoding for Kimi, and Ubicloud deployed it after validating 100% schema accuracy across its served model versions.