
Structured outputs: forcing LLM answers into JSON
In short: Structured outputs constrain an LLM's reply to match a JSON schema you supply, so code can parse it reliably even though values can still be wrong.
Structured outputs is an API feature that forces an LLM's answer into a JSON schema you supply, so required fields are present and types are correct. Anthropic's documentation, read in November 2026, says it guarantees schema-compliant responses through constrained decoding. The feature removes parsing failures and leaves reasoning errors untouched.
Structured outputs in plain words
It is like handing someone a form with labelled boxes instead of a blank page. They can still write the wrong answer in a box, but they cannot skip a box or write a poem.
How it works
You send a schema with the request. With constrained decoding, at each step the model may only produce tokens that keep the output valid against that schema. Anthropic's documentation describes the schema being compiled into a grammar, which adds latency the first time a schema is used and is then cached for 24 hours.
OpenAI's guide draws the line with the older JSON mode: both produce valid JSON, and only Structured Outputs ensures adherence to your schema.
Two exits remain. A model can refuse, which both vendors signal explicitly, and a response cut off by the token limit may be incomplete.
Example
from pydantic import BaseModel
class Invoice(BaseModel):
vendor: str
total: float
currency: str
due_date: str
result = client.responses.parse(
model="...", input=invoice_text, text_format=Invoice)
Your code gets an Invoice object, or an explicit refusal.
Where you'll meet it
- Extraction pipelines: invoices, contracts, forms, resumes
- Agent harnesses that need machine-readable plans
- Classification and routing steps
- The OpenAI and Anthropic APIs, and Claude on Amazon Bedrock, Google Cloud and Microsoft Foundry
What it changes for you
Developers can delete brittle regex parsing and retry loops. The work shifts to schema design and to validating values as well as shapes: a total that does not add up still passes the schema. For data teams, it makes LLM extraction behave like any other typed data source.
Structured outputs vs tool calling
Both use JSON schemas. Structured outputs shape the final answer; tool calling shapes a request for your code to act. Anthropic documents the two together, as JSON outputs and strict tool use.
Common mistake
My view: treating schema-valid as correct. A model forced into a schema will fill every required field, even when the source has no value, so it may invent one. OpenAI's guide warns of hallucinations when the input is unrelated to the schema, and shows a union with null as the way to make a field optional. Make fields nullable where data can be missing, and check values in code.
Related terms
Tool calling · Hallucination · Guardrails · Evals
From the archive: Constrained JSON does not constrain the tool choice
A schema-valid answer can still be wrong, so allow nulls and check the values in code.


