Validate JSON output and recover from failures

Continue the ticket task with exactly two fields, category and order_id. Allowed categories are billing, technical, and other. The category rules remain those from the Easy level: billing covers charges, invoices, and refunds; technical covers application errors and access failures; other covers everything else, too little information, or a ticket with both kinds of issue.

A valid order ID has the exact shape ORD- followed by three digits. Treat it as a complete identifier, so ORD-1234 does not contain a valid order ID for this exercise.

When no ID appears, order_id is null. When multiple distinct IDs appear, category is other and order_id is null. When exactly one distinct ID appears, it must be copied. Repeated occurrences of the same ID still represent one ID.

The following is an application-side JSON Schema. Adapt its supported subset separately for any provider generation interface.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "category": {"type":"string","enum":["billing","technical","other"]},
    "order_id": {"type":["string","null"],"pattern":"^ORD-[0-9]{3}$"}
  },
  "required": ["category","order_id"],
  "additionalProperties": false
}

In JSON Schema, naming a property does not require its presence. The required list establishes that obligation. Additional properties are allowed by default unless restricted. String constraints apply to string values, so this pattern does not invalidate null. Check the selected validator's schema-draft support.

A validated ticket classification can be stored or routed according to application policy. It does not itself authorize financial or account actions.

This exercise implements the small teaching contract explicitly. It is a focused alternative to importing a schema library. In a larger system, use a maintained validator and keep types aligned with the authoritative schema.

The function returns a discriminated result: the accepted ticket, or the first layer that failed.

type Ticket = {
  category: "billing" | "technical" | "other";
  order_id: string | null;
};

type ValidationResult =
  | { ok: true; value: Ticket }
  | { ok: false; kind: "parse" | "schema" | "source"; reason: string };

// Checks run in this order and stop at the first failure:
// parse:  raw is valid JSON
// schema: an object with exactly category and order_id, an allowed category,
//         and an order_id that is null or ORD- followed by three digits
// source: order_id follows the ID policy for the IDs found in source
declare function validateTicket(raw: string, source: string): ValidationResult;

TypeScript types are erased at runtime, so a type assertion alone would not validate external data. The function has to inspect the parsed value field by field before it can return a Ticket.

Read provider response metadata first. A refusal is a distinct response state. A truncated generation needs diagnosis using output limits and termination metadata. A network failure may leave completion unknown. Do not assume every response body is the requested JSON result.

For streaming, accumulate the complete result and validate it before committing application data. A partial preview can be displayed as provisional UI, but it should not trigger a downstream action.

Use a small attempt budget, such as two total generation attempts, plus a request deadline and output-size bound. These are teaching defaults, not universal production settings.

For a repairable formatting or schema error, send the original source, the intended contract, and a concise validation error. Revalidate the entire replacement response. A model-generated correction can introduce new mistakes.

A useful repair instruction is "The previous result used an unsupported category. Reclassify the original ticket using billing, technical, or other and return the required object." Keep untrusted prior output and source material labeled as data.

Do not repair missing facts by inventing them. A null or review state may be the correct result. Do not repeatedly retry a refusal to bypass its meaning. Retry temporary transport failures under a separate bounded policy. Ensure that overlapping retry layers cannot multiply attempts unexpectedly.

Deterministic normalization can be appropriate when explicitly part of the contract, such as trimming permitted surrounding whitespace. Silently converting an unknown category into a valid one can hide an error and change meaning.

Build fixtures for ordinary requests, absent fields, mixed categories, duplicate IDs, hostile instructions, and long inputs. Define expected labels before trying prompt variants. Keep a held-out set for confirming changes.

Measure parse validity, schema validity, source-policy success, category accuracy, repair frequency, and end-to-end success. Use explicit denominators. For example, schema validity among parseable outputs can look excellent while many responses fail parsing. Also report successful tasks divided by all attempted tasks.

Version the prompt and contract used in each run. Change one meaningful factor at a time where practical. Keep deterministic validator tests separate from repeated model evaluations.