LLM Structured Outputs: JSON Mode, Schemas and Validation

Reliable structured output from LLMs: JSON mode, schema-constrained decoding, validation and repair, retries, semantic checks and versioned contracts.

Intermediate 3 min read · Updated 29 October 2026

Whenever an LLM's output is consumed by code (an extraction pipeline, a router, a tool call, a form filler), it has to arrive in a structure your program can trust. Structured outputs combine three layers: asking for a format, constraining the model to it where the provider supports it, and validating the result in your code, with repair, retry and fallback paths when it fails. Interviewers probe this as "how do you integrate an LLM behind an application contract?"

Three levels of reliability

Level	How	Reliability
Prompted format	Describe the JSON or format in the prompt, with an example	Usually right, sometimes malformed or with extra text
JSON mode	Provider guarantees syntactically valid JSON	Valid JSON, but fields may be missing or wrong types
Schema-constrained decoding	Provider (or your serving stack) restricts tokens so output matches a JSON Schema	Matches the schema; values can still be semantically wrong

Even with schema-constrained decoding, the values can be wrong: an invoice total that doesn't equal the sum of its lines, a date in the future, a customer id that doesn't exist. Validation in code is always needed.

The validation pipeline

Treat model output like untrusted input: parse, validate against the schema, run semantic checks, and only then act. Failures get one bounded repair attempt, then a fallback.

Designing the schema

{
  "type": "object",
  "required": ["vendor", "invoice_date", "currency", "lines", "total", "confidence"],
  "properties": {
    "vendor":       { "type": "string" },
    "invoice_date": { "type": "string", "format": "date" },
    "currency":     { "type": "string", "enum": ["USD", "EUR", "INR", "GBP"] },
    "lines": { "type": "array", "items": {
        "type": "object", "required": ["description", "amount"],
        "properties": { "description": { "type": "string" }, "amount": { "type": "number" } } } },
    "total":        { "type": "number" },
    "confidence":   { "type": "string", "enum": ["high", "medium", "low"] },
    "missing_fields": { "type": "array", "items": { "type": "string" } }
  }
}

Enums over free text wherever values are a known set; easier to validate and route on.
Explicit unknowns: a missing_fields list or nullable fields, so the model isn't pushed to invent values.
A refusal path: a field or status for "can't do this" (wrong document type, unsafe request).
Keep it flat and small: deeply nested, very large schemas are harder for models and slower to constrain.
Version the schema alongside the prompt; consumers rely on it like an API contract.

Semantic checks before side effects

Arithmetic: line items sum to the total (within rounding).
References exist: the order id belongs to this customer, the product id is in the catalogue.
Ranges and dates are plausible: no negative quantities, no invoice dated next year.
Permissions: the action is allowed for this user, checked in code; see prompt injection.
Confidence routing: low-confidence or partial results go to human review instead of being auto-applied.

Function calling is structured output too

Tool calls are the model emitting a structured request (tool name plus arguments) that your code executes. The same rules apply: validate arguments against the tool's schema, check permissions and business rules, and make side-effecting tools idempotent. See tool calling and MCP.

Errors and retries

Failure	Response
Malformed JSON or schema violation	One repair attempt with the validation errors included; then fallback
Truncated output (hit max tokens)	Raise the limit or split the task; don't parse partial output
Refusal or "can't extract"	Route to a human or return a typed error to the caller
Timeout or rate limit	Retry with backoff on the same or a fallback model; see LLM gateway
Semantically wrong values	Reject, log the case into the eval set, route to review

Bound retries: each one costs tokens and latency, and repeated failures usually mean the input is unusual, not that the next attempt will work. Track the repair and fallback rates as quality metrics; see LLMOps.

Where it’s used

Document extraction

Invoices, receipts and contracts parsed into typed records with arithmetic and reference checks.

Routing and classification

Enum outputs decide which queue, model or workflow handles a request.

Agents

Every tool call is a structured output validated before execution.

In an interview

Layer requested format, constrained decoding and validation in code.
Design the schema with enums, explicit unknowns and a refusal path.
Run semantic checks before any side effect.
Bound repair retries and route persistent failures to a fallback.

Frequently asked questions

How do you get reliable JSON from an LLM?

Request the format, use schema-constrained decoding or JSON mode where available, then validate the output against the schema and run semantic checks in code, with a bounded repair retry and a fallback.

Is schema-constrained output always correct?

It guarantees the structure, not the values. Totals, ids, dates and permissions still need checks in code before any side effect.