Multimodal & Document AI · 2 to 3 hours
Validate page-region citations
Implement a validator for page-region citations and predict its result on fixed cases. Then design extraction for a mixed PDF with a rotated scan and a table split across two pages.
Implementation and pipeline design
3 h
Used in interview rounds:
Live Coding (Medium)1
Step 1: The question
Design extraction for the three-page handout-v2 upload in the design exercise: page and block records, the transform for the rotated scan, how the split table stays one table, review rules, and one evaluation measure per stage. Implement validateRegion (JavaScript) or validate_region (Python) to the stated contract. Explain what a passing validation establishes and what it leaves unproven.
Map a box from a rotated scan
Suppose page 1 of a fictional handout was scanned sideways and stored turned 90 degrees clockwise, and OCR ran on the stored image as it is. Coordinates are normalized, with the origin at the upper left of each image.
A point at (u, v) on the upright page appears in the clockwise-turned image at (x', y') = (1 - v, u): the page's top edge becomes the image's right edge. Inverting gives u = y' and v = 1 - x', so a box maps as:
Clockwise turn, image box to upright page box
[x0', y0', x1', y1'] -> [y0', 1 - x1', y1', 1 - x0']
| Step | Value |
|---|---|
| OCR box in the stored image | [0.75, 0.125, 0.875, 0.625], a tall strip near the right edge |
| Horizontal range on the page, from y' | 0.125 to 0.625 |
| Vertical range on the page, from 1 - x' | 1 - 0.875 = 0.125 to 1 - 0.75 = 0.25 |
| Box on the upright page | [0.125, 0.125, 0.625, 0.25], a wide strip near the top: one line of text |
Store the rotation with the page record and apply the inverse before saving evidence. Dividing pixel coordinates by the image size normalizes a box but leaves it in the turned frame. A counterclockwise turn needs a different mapping, derived the same way. Cropping adds an offset and scale, and deskewing a small rotation; undo them in reverse order.
The page-region validator
The validator checks one citation region against authoritative document metadata. The caller passes the expected revision and page count from the document record, never from the region itself.
type EvidenceRegion = {
documentRevision: string;
pageIndex: number; // zero-based physical page
box: [number, number, number, number]; // [x0, y0, x1, y1], normalized, upper-left origin
};
// Returns a new { documentRevision, pageIndex, box } with no other fields,
// or throws an Error with the message of the first rule that fails.
declare function validateRegion(
region: unknown,
expectedRevision: unknown,
pageCount: unknown
): EvidenceRegion;Python names the function validate_region, reads the keys document_revision and page_index, raises ValueError, and returns the four coordinates as floats.
| Order | Rule | Message when it fails |
|---|---|---|
| 1 | The expected revision is a non-empty string | Invalid expected revision |
| 2 | The page count is an integer of at least 1, not a boolean | Invalid page count |
| 3 | The region is an object (a dict in Python), not an array or null | Region must be an object |
| 4 | The region's revision equals the expected revision | Wrong document revision |
| 5 | The page index is an integer, not a boolean, with 0 <= index < page count | Invalid page index |
| 6 | The box is an array (a list or tuple in Python) of exactly four values | Expected four coordinates |
| 7 | Every coordinate is a finite number, not a boolean, string, NaN, or infinity. In Python it is an int or float that converts to a finite float | Coordinates must be finite numbers |
| 8 | 0 <= x0 < x1 <= 1 and 0 <= y0 < y1 <= 1 | Invalid normalized rectangle |
Booleans need an explicit check in both languages: Python's bool is a subclass of int, and JavaScript's Number(true) is 1. The result is built from the checked values, so fields the caller did not ask for never reach downstream code.
Constraints
- Follow the validator's rule order and error messages exactly
- Take the expected revision and page count from document metadata, never from the region
- Evidence regions use a zero-based page index and a normalized box with an upper-left origin on the upright page
- Use only the observations in the handout table; no better scan is available
Deliverables
- Page records with route, rotation, and status, a document status, and records for the split table and its footnote
- The upright-page box for the header row's OCR box, where the transform is stored, and what a passing validation does and does not establish
- `validateRegion` or `validate_region`, checked against every row of the prediction table
- One measure per stage: OCR, table structure, retrieval, and answering with citations
- Review rules for the uncertain cell, and how answers behave while it is open
2
Step 2: Hints
Optional. Each hint gives a little more away.
3
Step 3: Work through it
Code exercise
Page-region validator
Implement validateRegion (JavaScript) or validate_region (Python) to the contract above. Apply the rules in the stated order, throw (raise ValueError in Python) with the exact message of the first rule that fails, and return a new record with only the three contract fields.
Expected
The practice problem's fixtures use expected revision handout-v2 and page count 3. Its tests compare each returned record exactly (no extra fields; Python coordinates as floats) and each error by its exact message, so rule order matters whenever an input breaks more than one rule. Check your code against the prediction table below before you run them.
Limitations
The validator checks structure and identity only: one rectangle on one page, with no polygons or media time ranges. It trusts the caller's expected revision and page count, and production code should also cap payload size before validating. JavaScript cannot tell 1.0 from 1, so a page index of 1.0 passes in JavaScript but fails in Python, where it is a float.
Predict the outcome
Predict the validator's results
Use expected revision handout-v2 and page count 3 for every row. Inputs use JavaScript notation; in Python the keys are document_revision and page_index, NaN is float("nan"), and true is True. Write the returned record or the exact error message for each row before you reveal the answers.
| Row | Region revision | Page index | Box | Other input fields | Result |
|---|---|---|---|---|---|
| R1 | `handout-v2` | 2 | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R2 | `handout-v1` | 2 | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R3 | `handout-v2` | 3 | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R4 | `handout-v2` | 0 | [0, 0, 1, 1] | none | ? |
| R5 | `handout-v2` | 2 | [0.6, 0.2, 0.1, 0.25] | none | ? |
| R6 | `handout-v2` | -1 | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R7 | `handout-v2` | 2 | [0.1, 0.2, 0.1, 0.25] | none | ? |
| R8 | `handout-v2` | 2 | [0.1, NaN, 0.6, 0.25] | none | ? |
| R9 | `handout-v2` | 2 | [0, 0, 1.2, 0.5] | none | ? |
| R10 | `handout-v2` | `true` | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R11 | `handout-v2` | 2 | [true, 0.2, 0.6, 0.25] | none | ? |
| R12 | `handout-v1` | 5 | [0.1, 0.2, 0.6, 0.25] | none | ? |
| R13 | `handout-v2` | 2 | [0.1, 0.2, 0.6] | none | ? |
| R14 | `handout-v2` | 1 | [0.25, 0.5, 0.75, 0.625] | `label: "answer key"`, `verified: true` | ? |
Design exercise
Design extraction for a mixed handout
Suppose a coding-practice platform lets instructors upload exercise handouts as PDFs. Its study assistant answers learner questions and cites a highlighted region of a page. The current upload is revision handout-v2, with three pages.
| Page | What the file contains | What processing observed |
|---|---|---|
| 0 | Problem statement and a starter code block, with a text layer | Native extraction returns readable text that matches the rendered page |
| 1 | A scanned worksheet with no text layer, stored turned 90 degrees counterclockwise. The top half of the upright page holds the start of a test-case table: a header row (input, expected output, time limit) and rows 1 to 4. The bottom half is a hand-drawn diagram | OCR on the stored image places the header row at image box [0.125, 0.25, 0.25, 0.875]. Row 3's expected output reads as 1O or 10, with low confidence |
| 2 | Rows 5 to 8 of the same table with no header row, then a footnote: "All time limits are in milliseconds." Has a text layer | Native extraction returns the rows as tab-separated lines and the footnote as a paragraph |
A learner asks: "What is the time limit for test case 6?"
Requirements
- Records per page (route, rotation, status) and for the document as a whole
- Records for the table and the footnote, each with evidence in upright page coordinates
- The transform for page 1 and the upright-page box for the header row
- A plan to answer the learner's question, with every region it cites
- Review rules for low-confidence or conflicting values
Constraints
- Evidence regions follow the validator's shape: revision, zero-based page index, and a normalized box with an upper-left origin on the upright page
- Use only the observations in the table; no better scan is available
Deliverables
- A page table and a document status
- Example records for the logical table and its two segments (JSON or a table)
- The transform, the mapped header box, and the arithmetic
- The answer plan for test case 6 with its citations, and the review rules
Kept in this browser. Saving your score below keeps a copy with it.
0 / 5000
4
Step 4: Reference answer
5
Step 5: Score yourself
Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.
Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.
0 of 10 · Scored 0 of 5 dimensions
Ready at 8 of 10, with no 0 in Evidence and coordinate mapping or Failure and review behavior
Kept in this browser until you sign up, then it moves to your account.
Complete this level when your validator passes every prediction row and you can trace a displayed field back through each transformation to its region on the original page.
Your saved scores
Practice saves and mock interview answers on this question, newest first.
No saved scores yet. Score every dimension, then save.
6
Step 6: Follow-up questions
An interviewer may ask these next.
Why not always use OCR?
Why version extraction separately from the source document?
Can normalized boxes handle a rotated crop automatically?
What does schema validation miss?
Why does the validator take the expected revision and page count as arguments instead of reading them from the region?
7
