Interview Practice
AI System Design
Design an AI-powered system with explicit guarantees, evaluation, and operations.
What interviewers evaluate
- Explicit guarantees and data flow
- Security boundaries
- An evaluation plan
- Cost and latency budgets
- Incident recovery
What you will be asked
Design a support assistant with citations and measurable quality.
In the room
Requirements, architecture, data flow, an evaluation plan, and the main failure modes. Diagrams and pseudocode are fine.
Timebox
45-60 minutes
Mock interview
A timed answer to a random question from this level, then the reference answer and your score.
How to practice
- Start with the linked design question; the other two practice missing evidence and a scored evaluation set.
- Follow the outline: requirements, architecture, data model, failure modes, evaluation, and operations.
- With a peer: they play the product manager and push on cost, latency, and wrong answers.
What a strong answer shows
- Starts from requirements and success metrics
- Builds citations from trusted source mappings, not from model output
- Has an evaluation set with answerable, multi-passage, and unanswerable questions
- Plans for document updates and deletion
- States cost and latency budgets
Questions for this level
Each question opens on its Practice tab, and Previous and Next follow this list.
RAG & Data Pipelines
3 h
Design a support assistant that answers from help articles with citations the customer may see, or hands off to a human. Turn its requirements into metrics, plan its evaluation set, keep it fresh through updates and deletion, and check its token, cost, and latency budgets.
RAG & Data Pipelines
3 h
Trace a missing policy exception through ingestion, eligibility, and context packing without weakening access rules.
Evaluation & Observability
3 h
Score a six-case evaluation set with a deterministic harness, report success, coverage, and answered accuracy with named denominators, and keep missing outputs visible in the totals.
Follow-up questions
An interviewer may ask these next.
- How do you prevent a citation to a document the user cannot see?
- What is your no-answer behavior, and how do you measure it?
- How do you roll out a prompt change safely?
Scoring rubric
Interview ready at 8 of 10 with no 0.
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Problem framing | Jumps to a solution or solves the wrong problem | Restates the problem but misses key constraints or success criteria | Clarifies goals, constraints, and success criteria before solving |
| Correctness | Key claims or code are wrong | Mostly right with gaps or unhandled cases | Correct and precise, including edge cases |
| Evidence or testing | No way to verify the answer | Generic testing ideas | Concrete fixtures, metrics, or traces that would confirm or refute the approach |
| Trade-offs | One option, no alternatives | Names alternatives superficially | Compares options against the stated constraints and says what would change the decision |
| Communication | Disorganized or unclear | Understandable but meandering | Structured and concise; states assumptions; checks in with the interviewer |