Valid JSON, False Data: Five Pitfalls and What to Do

Impeccable structured results can mask substantial errors. Assessments attributed to BAML and academic work report a decrease in accuracy under format constraints and describe five failures that schema validation does not catch. A three-layer defense and, in some cases, resampling are proposed to enhance reliability.

Three Lines of Defense and Numeric Signals to Activate Them

A three-layer approach is proposed to strengthen the reliability of structured outputs. The first layer relies on schema and structural validation using tools like Pydantic, JSON Schema, or Zod, which detect type errors, missing fields, and invalid enumerations. The second layer adds inter-field semantic validators, value entropy tracking, and periodic audits to cover the five modes of failure that concern meaning rather than form. The third layer involves adding an optional confidence field to each extracted value, allowing the model to indicate its uncertainty. According to Cleanlab's CONSTRUCT benchmark, field reliability scoring better detects errors in the structured outputs of GPT-5 and Gemini than confidence estimates at the prompt level. On the generation side, BAML reports that parse-and-retry outperforms constrained decoding by more than 2 accuracy points on the same model, at the cost of a success that is not guaranteed on the first pass; the relevance of this high-volume exchange, where latency increases with repetitions, remains to be evaluated. These measures respond to a finding: under schema constraints, uncertainty is masked. When a value is required, the model provides it, whether it is grounded or not, and fields like risk_score or summary consistently receive content. There is no standard mechanism to signal "I don't know" or "not applicable," which allows poor responses to emerge with the same apparent confidence as good ones.

Constrained Formatting Costs Accuracy, According to Several Studies

Measurements attributed to Lee et al. show that, on open-weight models, imposing a structured format reduces accuracy by 3 to 9 points, with a gap exceeding 15 points on mathematical reasoning tasks. Tam et al. observe an increased degradation of reasoning as formatting rules become stricter. BAML notes that, on function call tasks, there is 93.63% accuracy in free generation with post-hoc analysis compared to 91.37% in constrained decoding, showing that a JSON that is always valid can be less accurate than a JSON that is sometimes invalid. These results occur in a context where XGrammar has become the default backend for vLLM and TensorRT-LLM, with a token cost close to zero and a largely resolved syntax issue. However, there remains a less visible cost: part of the model's attention is dedicated to adhering to the format rather than the content, which teams do not always take into account.

Five Types of Errors Persist Despite Correct Formats

Five modes of errors persist despite schema-compliant outputs. Enumeration hallucination involves choosing a valid but contextually inappropriate value. Confident fabrication fills text fields with plausible but invented data. Contradictions between fields pass unit checks while rendering the record impossible. Distributional collapse pushes the model towards default values deemed safe. Table hallucination replaces an empty table with fabricated items.

Concrete Examples: Priorities, Phantom Expenses, Impossible Dates

With a priority enumeration like [“low”, “normal”, “high”, “urgent”], the grammar guarantees a value but not its relevance, and the model may classify a routine request as “urgent.” BAML reports that by submitting a photo of an elephant as a receipt, constrained decoding produced a complete and valid expense report according to the schema, with no possibility for the model to refuse or express doubt. Field-by-field checks do not detect global inconsistencies: a sentiment excerpt {sentiment: “positive”, score: 0.1} or a date range with an end date earlier than the start passes unit validation. The bias towards probable tokens favors generic values like 0.95, “medium,” or “general.” Finally, since empty tables are of low probability under grammar, fictitious items appear and create “phantom matches” when the correct answer is zero.

Operational Monitoring: Distributions, Entropy, and Inter-Field Rules

Several signals facilitate the detection of failures. For enumeration hallucination, it involves tracking the distribution of values by field over time. For confident fabrication, it is necessary to monitor the absence of refusals or null values and audit outputs from ambiguous inputs. Against contradictions, Pydantic validators with inter-field logic are recommended. For distributional collapse, monitor entropy and alert when it retracts. For tables, compare the observed rate of empty tables to the expected baseline rate. Writing more rules can help on known models, through checks like start_date > end_date or score-label consistencies, but faces the open diversity of semantic errors. An alternative is resampling: generate freely, validate, and regenerate in case of failure. BAML reports that parse-and-retry exceeds constrained by more than 2 points on the same model. In all cases, schema compliance remains a baseline to be supplemented by semantic checks, implemented after repeated incidents.