Structured output with tool use and JSON schemas

Tool use is the reliable path

Asking for JSON in prose and parsing it is a recurring source of syntax errors. Strict tool use is the reliable approach: define the input schema, set strict: true on the tool, and choose a tool-choice mode that requires the call. The resulting tool name and input are schema-validated, so downstream code doesn't have to repair almost-JSON.

Repairing almost-JSON with a tolerant parser is the tempting shortcut, and it is a guess. A parser that silently closes a brace or drops a stray token can silently change which field a value landed in, and nothing downstream records that a repair happened.

A non-strict tool schema guides generation but doesn't make that guarantee. Also inspect the response status: a refusal, a token limit, or any other stop reason must be handled before treating a response as a successful extraction.

Check the stop reason before you read anything out of the response:

if (response.stop_reason !== 'tool_use') {
  // refusal, max_tokens, or a plain text answer — nothing was validated
  return handleIncomplete(response.stop_reason);
}
const call = response.content.find((block) => block.type === 'tool_use');
const invoice = call.input; // schema-validated only on this path

if response.stop_reason != "tool_use":
    # refusal, max_tokens, or a plain text answer — nothing was validated
    return handle_incomplete(response.stop_reason)
call = next(b for b in response.content if b.type == "tool_use")
invoice = call.input  # schema-validated only on this path

The guarantee attaches to a tool call, not to the response. Reading input off a response that stopped for any other reason is reading a field that is not there.

The three tool_choice modes

• auto — the model may call a tool, or may just answer in text. Fine when a tool is optional; useless as a structured-output guarantee.
• any — the model must call a tool, but chooses which. This is the setting when several extraction schemas exist and you do not know the document type in advance.
• Forced, {"type": "tool", "name": "extract_metadata"} — a specific named tool must be called. Use it when a particular extraction has to run, for instance before an enrichment step that depends on it.

What schemas do not validate

A strict schema eliminates syntax errors. It does nothing about semantic ones.

Line items that do not sum to the stated total, a value placed in the wrong field, a date that parses but is wrong — all of these are perfectly schema-valid. Validation of meaning is a separate job, and 4.4 is where it lives.

Optional fields prevent fabrication

If a field is required and the source document does not contain it, the model must still return a value. That can lead it to invent one. Mark fields optional or nullable when the source may genuinely omit them.

You've met a form that won't submit until every box is filled, including boxes that don't apply to you. You typed something anyway. A required field puts a model in that position, and it will type something too.

An empty string isn't a substitute. It's a value, so downstream code can't tell a field the source omitted from one the source left blank, and the two mean different things to anyone reconciling the record later. Absence has to be representable as absence.

Enums need the same escape hatch. Add unclear for ambiguous cases, and other paired with a detail string for categories you didn't anticipate. Here are all three moves in the properties block of one extraction tool, on a tool declared with strict: true:

"properties": {
  "purchase_order": { "type": ["string", "null"] },
  "category": { "enum": ["hardware", "services", "other"] },
  "category_detail": { "type": "string" }
}

purchase_order accepts null, so an invoice that never carried one can say so instead of inventing a number. category keeps its enum, and other paired with category_detail is where an unforeseen category lands without being forced into hardware.

Dropping the constraint and accepting free text is the other way to stop misclassification, and it costs the guarantee that made the field usable: every consumer downstream now has to interpret arbitrary strings. Keep the enum and add the escape value instead.

Without them, every unforeseen case gets forced into the nearest listed value and the data silently misrepresents the source.

Normalization belongs in the prompt

Schemas constrain shape, not formatting. If sources express dates or units inconsistently, state the normalization rules in the prompt alongside the strict schema.