Inference & Serving · Fast-moving · Intermediate

Grammar-Guided Structured Decoding

Also known as: Grammar-Constrained Decoding, Regex LLM Output

An inference-time technique that restricts model token logits to enforce strict JSON, XML, or code syntax matching a defined grammar.

What Grammar-Guided Structured Decoding is

Grammar-Guided Structured Decoding guarantees 100% syntactically valid outputs without retry loops or post-parsing errors.

How it works

Constructs a context-free grammar state machine that masks out invalid tokens before computing final softmax distributions.

Why it matters

Widely used for function calling, API integration, and structured database extraction.

Common uses