Validate a model's JSON review summary

A reply is accepted only when it is a JSON object with exactly these three keys:

The validator runs these checks in this order:

• Steps 1 and 2 end validation at once: the result holds only that code.
• Steps 3, 4, and 5 each add at most one code: the first rule in the listed order that fails for that field. All three fields are always checked.
• Field codes come first in contract order (rating, sentiment, topics), then the extra codes, sorted by plain string comparison (JavaScript's default sort(), Python's sorted()). Only the extra codes are sorted.
• No code means the reply is accepted.

The result, in TypeScript terms:

type ReviewSummary = {
  rating: number; // an integer from 1 to 5
  sentiment: "positive" | "mixed" | "negative";
  topics: ("price" | "delivery" | "condition" | "story")[];
};

type ValidationResult =
  | { ok: true; value: ReviewSummary }
  | { ok: false; errors: string[] };

declare function validateReviewSummary(raw: string): ValidationResult;

Python's validate_review_summary(raw) returns a dict with the same keys, using lists, never tuples. An accepted value is a new object with its keys in contract order, rating as an integer (a Python int, also for an input such as 4.0), and topics in the reply's order. A failed result has no value key, and an accepted one has no errors key.

A TypeScript type or a Python annotation checks nothing at runtime: after parsing, a field can hold anything JSON allows, and the two parsers disagree on some types.

• In JavaScript, typeof returns "object" for null and for arrays, so a plain-object check also needs value !== null and !Array.isArray(value). typeof 4.5 is "number", so use Number.isInteger, which rejects fractions, strings, and booleans.
• In Python, bool is a subclass of int, so isinstance(True, int) is True and True equals 1. Reject booleans before you check for an integer.
• The contract defines an integer as a number with no fractional part, as JSON Schema does, so both languages accept 4.0. In Python, float.is_integer() tests it and int(4.0) gives the value to return.
• A key with the value null is present; an absent key is missing. Separate codes keep the causes apart: the model left the field empty, or it dropped the field.

JSON Schema allows properties a schema does not list, unless additionalProperties is false. For model output, rejecting them is the safer default:

1. A field nobody reviewed, such as a reply_to_customer the model invents, could otherwise reach storage or the dashboard.
2. A new key signals that the prompt or the model changed.
3. A key in the wrong letter case shows up twice, as a missing field and as an extra key, which points at the cause.

Adding a field later is a contract change: the prompt, the validator, and every consumer change together under a new contract version (the Hard level, "Design a versioned output contract and migration").

Each error is a short code shaped <subject>:<rule>, such as rating:range, whose wording never changes. Tests assert exact codes, logs count them per prompt and contract version to show which rule the model breaks most, the dashboard maps each one to a message for moderators, and a later corrective request to the model could quote them. Prose messages change with every edit and break whatever reads them.

Every failing field, in one pass. The three fields are independent: a bad rating says nothing about the topics. So one pass reports each failing field once, with its first failing rule, and one log line shows everything wrong with a reply. A failure of the whole reply still stops at once, because there is no object to check.

Compared with the Medium level. "Validate JSON output and recover from failures" checks a ticket reply in dependent layers, so it stops at the first failing layer and then designs a bounded retry; this level has one structural layer and reports every failing field.

Bad model output is an expected input, not a bug in your program, so the validator returns codes instead of throwing.

Apply the contract by hand before you write any code. Each reply is the exact text the validator receives (R2 is described in words because it contains a code fence). The practice problem's first test lists the expected results, so predict R1 to R8 before you open it.

In a live-coding round, settle the contract before the code. Restate it (the three fields, the two result shapes, the order of the codes), then ask about the unclear cases: is null the same as missing, may topics be empty, does letter case matter, is 4.0 an integer, and what happens to an unknown key? This contract answers all five; in an interview, ask. Then write the tests as a table of replies and expected results, before the function:

Keep the tests readable as the contract grows: one group per field, test names that state the rule ("a rating of 0 is out of range"), and a helper that builds a valid reply and changes one field, so each row shows only what differs.

Implement the contract in JavaScript or Python. Write your test table first, then the function, and run the practice problem's tests.

A teammate's validator is easiest to review with the fixtures in hand: run each reply through their code in your head and compare the result with the contract.

A reply can pass every rule and still misread the review: sarcasm ("Great, the book arrived in pieces") summarized as positive, story on a review that mentions only the price, or a rating of 1 next to positive sentiment. The structure is right, so the validator accepts each one. Other checks catch them: an evaluation set of labeled reviews (the Evaluation and Observability levels), moderator spot checks, or a consistency rule the contract states explicitly, such as the ratings each sentiment allows, which then becomes one more code.

Validate every reply before it is stored or displayed, and log each rejection's codes with the prompt and contract version. Asking the model again is a separate choice with a small attempt budget and a deadline; the Medium level designs it.

Provider structured outputs reduce parse and type failures but do not remove them: one provider documents that refusals and token-limit cut-offs can miss the schema, that enum capitalization can differ, and that minimum and maximum are not enforced. This contract compares sentiment exactly, by choice; declaring lowercase normalization before coding is also valid.