Multimodal & Document AI · 3 to 4 hours
Design a verifiable document and media pipeline
Design a pipeline that turns a partly scanned course pack and a lesson recording into cited study material, tracking coverage, transforms, and cross-modal conflicts, and decide what it may answer while processing is still partial.
4 h
Used in interview rounds:
AI System Design (Hard)Overview
Design a service that answers questions over mixed documents and media while keeping every claim verifiable. A larger model cannot recover evidence that was never captured or rebuild a discarded source reference, so the work is controlling loss and uncertainty at every stage. You practice on a partly processed course pack and lesson recording.
Study time: 3 to 4 hours
Interview format: System design discussion with a coverage and failure walkthrough
By the end you can
- Define job identity, terminal states, and coverage accounting so a partly processed source states exactly what it can support
- Link every claim to page regions or recording time ranges through extraction revisions and recorded transforms
- Handle disagreements between audio, slides, and pages with an explicit rule and full provenance
- Set escalation budgets and review triggers from measured failure signals and stratified evaluation
- Keep instructions inside sources away from permissions, and enforce current access on every derivative
- Write failure-injection tests that check coverage, provenance, and access invariants
Prerequisites
Helpful background: Familiarity with job queues, idempotent workers, and basic coordinate transforms helps.
This topic covers routing uncertain inputs, page-region and time-range evidence, and document and media pipelines. RAG & Data Pipelines covers generic ingestion, chunking, and the document lifecycle.
In this guide
- Define the product contract
- Durable processing architecture
- Idempotency and partial completion
- Evidence as a graph
- Coordinate and timeline integrity
- Cross-page and cross-modal reasoning
- Budgets and selective automation
- Security boundaries across modalities
- Evaluation design
- Operations, release, and migration
- Scenario: a course pack and a lesson recording
- Design the ingestion pipeline
- Failure-injection matrix
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.
Related on CodeSnatch
Lessons, problems, and articles that build the skills this question uses.
System Design
Splits long media into chunk jobs with deterministic keys and marks a video ready only after every chunk is done.
AI Engineering
Shows how citations, caches, and access checks stay correct through revisions, retries, and permission changes.
System Design
Pairs at-least-once delivery with idempotent consumers and dead letter queues, as redelivered page jobs need.
AI Engineering
Practice reconciling a derived index against its manifest after a partial backfill leaves it inconsistent.
System Design
Explains append-only event logs, projections that build read models, and when event sourcing is not worth its cost.
Practice the interview question
Answer it in your own words before you look at the reference answer.