Prompting & Structured Output · 2 to 3 hours
Validate JSON output and recover from failures
Implement a dependency-free validator for a ticket contract, classify each fixture failure by layer, and design a two-attempt recovery that never commits invalid output.
3 h
1
Step 1: The question
Implement the ticket contract, classify the fixture failures, and propose a maximum-two-attempt recovery process. Explain which output the validator cannot prove correct.
Start with a precise application contract
Continue the ticket task with exactly two fields, category and order_id. Allowed categories are billing, technical, and other. The category rules remain those from the Easy level: billing covers charges, invoices, and refunds; technical covers application errors and access failures; other covers everything else, too little information, or a ticket with both kinds of issue.
A valid order ID has the exact shape ORD- followed by three digits. Treat it as a complete identifier, so ORD-1234 does not contain a valid order ID for this exercise.
When no ID appears, order_id is null. When multiple distinct IDs appear, category is other and order_id is null. When exactly one distinct ID appears, it must be copied. Repeated occurrences of the same ID still represent one ID.
Separate validation layers
| Layer | Question | Failure example |
|---|---|---|
| Response state | Did generation complete with a usable result? | Refusal or interrupted stream |
| Parsing | Is there exactly one usable JSON document? | Trailing comma |
| Structure | Are fields, types, and enums valid? | category is urgent |
| Source checks | Are copied values supported by the input? | Invented order ID |
| Task semantics | Does classification follow the policy? | An invoice request labeled other |
| Application policy | Is this operation permitted? | A classification used to authorize a refund |
A validated ticket classification can be stored or routed according to application policy. It does not itself authorize financial or account actions.
Constraints
- The contract has exactly two fields, category and order_id, with the rules in the contract section
- At most two generation attempts per request, within one request deadline
- No downstream action runs before a result is accepted
- Repairs never invent a value the ticket does not support
Deliverables
- A validator that returns the accepted ticket or the first failing layer: parse, schema, or source
- The expected result for each of the seven fixtures
- A recovery process of at most two attempts, including what happens when it ends
- The fixture the validator accepts but cannot prove correct, and why
2
Step 2: Hints
Optional. Each hint gives a little more away.
3
Step 3: Work through it
Code exercise
Ticket output validator
Implement validateTicket(raw, source) for the ticket contract. Return ok with the parsed ticket, or the first failing layer: parse when the text is not JSON, schema when the fields, types, or values break the contract, and source when order_id does not follow the ID policy for the ticket text.
Limitations
This validator implements one small contract by hand. It does not read provider response states, reject duplicate JSON keys (JSON.parse keeps the later value, so use a parser that detects duplicates if they must be rejected), or handle Unicode and punctuation rules a production identifier may need. In a larger system, use a maintained schema validator for the structural layer.
Predict the outcome
Classify the worked fixtures
Run each raw output through the layers in order: parse, structure, then source checks. Write the first failing layer, or say that it passes, before you reveal the expected results.
| Source | Raw output | Expected result |
|---|---|---|
| Refund ORD-204. | {"category":"billing","order_id":"ORD-204"} | ? |
| App crashes. | {"category":"technical","order_id":null,} | ? |
| App crashes. | {"category":"urgent","order_id":null} | ? |
| App crashes. | {"category":"technical"} | ? |
| App crashes. | {"category":"technical","order_id":"ORD-204"} | ? |
| Refund ORD-101 and ORD-202. | {"category":"billing","order_id":"ORD-101"} | ? |
| Refund ORD-204. | {"category":"technical","order_id":"ORD-204"} | ? |
Kept in this browser. Saving your score below keeps a copy with it.
0 / 5000
4
Step 4: Reference answer
5
Step 5: Score yourself
Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.
Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.
0 of 10 · Scored 0 of 5 dimensions
Ready at 8 of 10, with no 0 in Semantic awareness
Kept in this browser until you sign up, then it moves to your account.
Complete this level when you can reproduce the fixture outcomes, explain the validator's limits, and show how retries terminate without committing invalid output. Readiness also expects full credit (2) on semantic awareness: distinguishing structural from semantic correctness.
Your saved scores
Practice saves and mock interview answers on this question, newest first.
No saved scores yet. Score every dimension, then save.
6
Step 6: Follow-up questions
An interviewer may ask these next.
Why keep runtime validation with TypeScript?
Why not fix every malformed response with regex?
When should a retry stop?
Can schema-constrained generation replace business checks?
Why keep the original input during repair?
7