Validate page-region citations

Separate ingestion, extraction, normalization, validation, indexing, and answering. Each stage should produce a versioned result with explicit success, partial-success, and failure states. Avoid turning a missing page into an empty successful result.

For each document, retain an immutable revision identifier, original content hash, media type, tenant or access scope, page count, and processing configuration. Hashes help identify content; they do not grant authorization or establish who created it.

For every page, store processing status, dimensions, coordinate convention, rotation, and the extraction method used. A mixed PDF may have native text on one page and OCR output on another.

Attempt native extraction when appropriate, then assess its usefulness. Empty text, invalid character patterns, or a major mismatch with visible content can trigger OCR or review. A nonempty text layer is not enough to trust it.

Use layout analysis for multi-column pages and tables. Use visual models for diagrams or regions whose meaning is not represented by extracted text. Preserve competing outputs when they disagree instead of silently treating the latest response as correct.

Define a processing budget: maximum pages, image dimensions, audio duration, retries, and escalation cost. If limits are reached, report which parts were processed and which remain unavailable.

Represent a document as pages containing typed blocks: paragraphs, headings, tables, figures, code regions, and selection marks. Give each block a stable identifier within a specific extraction revision.

Store raw text separately from normalized text. Record the mapping between derived content and source regions. When OCR is rerun, block boundaries and IDs may change; do not attach old citations to the new extraction without a verified mapping.

Tables need cells with row and column positions, spans, text, and evidence. Captions and footnotes should remain linked to the objects they qualify. Provider layout services can expose these elements, but a canonical application representation avoids binding downstream code to one response schema.

Coordinates may be pixels, physical page units, or normalized fractions. Origins and rotation conventions vary. Declare these explicitly and convert at the boundary.

This guide uses zero-based physical page indexes and normalized rectangles [x0, y0, x1, y1] with an upper-left origin on the canonical upright page. A rectangle encloses a region but may include unrelated content; polygons can preserve finer geometry.

If OCR runs on a cropped, rotated, or deskewed image, retain the transform needed to map its output back to the original page. Normalization alone does not undo cropping, rotation, or perspective changes.

The validator checks one citation region against authoritative document metadata. The caller passes the expected revision and page count from the document record, never from the region itself.

type EvidenceRegion = {
  documentRevision: string;
  pageIndex: number; // zero-based physical page
  box: [number, number, number, number]; // [x0, y0, x1, y1], normalized, upper-left origin
};

// Returns a new { documentRevision, pageIndex, box } with no other fields,
// or throws an Error with the message of the first rule that fails.
declare function validateRegion(
  region: unknown,
  expectedRevision: unknown,
  pageCount: unknown
): EvidenceRegion;

Python names the function validate_region, reads the keys document_revision and page_index, raises ValueError, and returns the four coordinates as floats.

Booleans need an explicit check in both languages: Python's bool is a subclass of int, and JavaScript's Number(true) is 1. The result is built from the checked values, so fields the caller did not ask for never reach downstream code.

Apply the rules by hand first; the rows then serve as your test set.

For each extracted field, keep the raw value, normalized value, status, and evidence list. Normalize only with an explicit rule. For example, 03/04/2026 is ambiguous without a known date convention; silently choosing a month changes meaning.

Distinguish absent, unreadable, conflicting, and not-applicable values. Validation should check both syntax and relationships: a JSON number can still have the wrong unit, and a plausible total can belong to the wrong row. Use deterministic arithmetic for totals and unit conversions, and route a mismatch to review instead of overwriting source values to make them reconcile.

Chunk by meaningful structure while preserving evidence links. A table row may need its column headers and units repeated in the retrieval representation. A code block may need the preceding exercise instruction. Do not split these relationships merely to achieve equal character counts.

Text embeddings cannot retrieve what transcription omitted; visual retrieval can help locate a diagram but does not provide exact field values by itself.

Store access metadata with every derivative and filter retrieval by authorized scope. A shared embedding index must not become a way to search another tenant's source content.

Media evidence is a time range instead of a box. Store transcript segments with start and end times in a declared unit, the media revision, and any offset introduced by trimming. Align speech and screen frames on that one timeline: a spoken "this line" needs the frame visible at that moment. Reconcile overlapping windows by time range rather than appending every result, and keep uncertain speaker labels apart from verified identities.

Character error rate is the edit distance divided by the reference character count under a stated normalization (empty references, punctuation, casing, whitespace). For code, score operators and indentation separately where they change meaning.

Show the source region beside the extracted field. Let reviewers correct the value, evidence, and uncertainty independently. Preserve the original model output and the reviewer action for audit and evaluation.

Use review outcomes to locate systematic errors by document family, language, quality, and layout. Do not automatically turn every correction into training data without checking authorization, provenance, and split contamination.

This exercise applies the earlier sections to one upload. It is the scenario for the canonical question.