Structured outputs
Constrain model output to a schema so downstream systems can consume it reliably.
Draft
Model interaction
Draft page
This page is an outline. It describes what will be covered and is not yet complete technical documentation.
Asking a model to reply in JSON and hoping for the best does not survive production traffic. Structured output support constrains generation so the result conforms to a schema by construction.
This page covers the available approaches and their costs.
What you will learn
- The difference between prompting for a format and constraining generation to it.
- How grammar and schema-constrained decoding work at a high level.
- What constrained decoding costs in latency and flexibility.
- How to handle the cases a schema cannot express.
Recommended outline
This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.