Structured Outputs: Valid JSON, False Data Without Safeguards

A transaction extraction pipeline showed 2 to 3% discrepancies despite flawless JSON responses. The identified cause: required fields filled by the model in the absence of information, often with the execution date. Concrete solutions are emerging: making fields nullable, requiring textual proof, and validating on the code side with limited retries.

Business Validation: Pydantic Checks and Two Maximum Retries

Some errors escape the schemas, such as a negative amount or a future date. Additional instructions added to the prompt proved insufficient to enforce these rules. A Pydantic validator, on the other hand, applies deterministic constraints, such as requiring a positive amount and prohibiting a transaction dated in the future. In the observed cases, negative amounts appeared exactly twice, as the text described refunds. The generation API ensures structure, while Pydantic checks business consistency during parsing, without involving the model. In case of failure, two paths exist: directing to a human or an automatic retry. An operational strategy returns the detailed error to the model with a strict limit of two attempts, providing the previous raw output and asking to correct only the erroneous field. Extending retries indefinitely is discouraged: after two failures, the problem almost always originates from the source document, and an additional attempt consumes API calls for a case that a human could resolve quickly.

A Dedicated Field Allows Indicating the Exact Source of Each Data Point

Even with optional fields, a returned value may not explicitly come from the text. To trace the origin, a sibling field containing the exact source excerpt is added next to each value, via a generic wrapper combining value and evidence. This choice replaces dedicated types with a union and reduces type safety, but becomes cost-effective when the schema exceeds a few field types. The order of evidence followed by value constrains the model to cite before responding, and provides reviewers with a verifiable element. A value without proof, or supported by text absent from the source, materializes hallucination in the data. This setup has a cost: over a few hundred messages, the output token volume increased by about one-third and latency noticeably rose. It can be avoided for trivial fields, such as a five-digit postal code, and retained for financial amounts requiring action.

When the Schema Forces Inventions: The Case of Missing Dates

After three weeks of using Structured Outputs on payment confirmations, a reconciliation revealed a discreet but persistent flow of anomalies, around 2 to 3% of the weekly volume. The lines were neither broken nor malformed: the amount and sender matched, only the date differed. The hypothesis of a timezone issue was ruled out. Each case originated from a message that did not contain a date, for example, "Payment received from Chinedu, ₦45,000, ref TXN-82K91." The model, however, filled in transaction_date, almost always with the job execution date, within an hour's difference. The cause was a schema imposing a required date, which pushes the model to provide an acceptable value for typing. Considering a valid JSON as the end result masks this silent failure: the absence of a real value only becomes apparent when a downstream process depends on it. Structured Outputs address the risk of invalid JSON, not the reality of the fields.

Nullable Rather Than Default Inference: Making Absence Explicit

An empty field often reflects the truth of the sources. Making fields nullable allows the model to explicitly indicate the absence of information, rather than inferring a value. The confusion between extraction—reporting the text exactly—and inference—deducing what it implies—arises particularly from required fields without data, such as an ISO date when the message simply states "Tuesday." By making fields optional, the decision to infer returns to the application code, which can, for example, request additional information when the date is absent.

What Structured Outputs Provide, and What They Do Not

Native schemas and modern APIs have simplified obtaining clean JSON, where regex parsers, retry loops, and strict prompts were previously necessary. A Pydantic model with explicit types combined with a direct parsing call produces valid objects from a well-formed document—such as one with "Date: August 11, 2026"—without managing format artifacts. However, when required fields are missing, the model may insert a plausible value that passes type checks without reflecting the source. These limitations and the detailed remedies are not specific to any given provider. The code can then log the transaction ID after a successful parsing, in an operational flow free from format errors but not, on its own, from content errors.