Interview Practice
AI System Design
Design an AI-powered system with explicit guarantees, evaluation, and operations.
What interviewers evaluate
- Explicit guarantees and data flow
- Security boundaries
- An evaluation plan
- Cost and latency budgets
- Incident recovery
What you will be asked
Design an AI system with explicit guarantees, evaluation, and operations.
In the room
One open design prompt, such as multi-tenant RAG, a private agent, side effects, SLOs, output contracts, or an agent harness. The interviewer probes guarantees, failure handling, and trade-offs.
Timebox
45-60 minutes
Mock interview
A timed answer to a random question from this level, then the reference answer and your score.
How to practice
- Pick one linked question per session. Write your design notes before you open the study guide, then compare.
- For every component, write the guarantee it gives and the failure it must survive.
- With a peer: they inject a failure (a crash, a revoked permission, or a slow provider) at any point and you trace its effect.
What a strong answer shows
- Explicit guarantees and where each is enforced
- Security boundaries for data and tools
- An evaluation plan with stage metrics and release gates
- Cost and latency budgets with numbers
- Recovery from crashes, retries, and partial failures
Questions for this level
Each question opens on its Practice tab, and Previous and Next follow this list.
RAG & Data Pipelines
4 h
Design a private multi-tenant policy assistant whose answers and citations stay correct through asynchronous updates, retries, permission changes, and deletion.
Security & Privacy
4 h
Threat-model a multi-tenant assistant that reads private documents, keeps conversation summaries, and sends approved notifications, naming where each control is enforced and what risk remains.
Tool Calling & Agents
4 h
Design an agent that closes a ticket and sends one approved closure email safely across duplicate jobs, crashes, timeouts, and concurrent workers, and say exactly what the design can and cannot guarantee.
Production & Reliability
4 h
Design reliability controls for a multi-tenant AI assistant: SLOs and error budgets, quota-aware admission, fallback and cost reservations, staged rollout with a canary decision, rollback, and recovery.
Prompting & Structured Output
4 h
Upgrade a ticket classifier to a versioned contract with multiple IDs, evidence, and review states, keep two v1-only consumers working, and decide from evaluation data whether a candidate prompt should ship.
Embeddings & Retrieval
4 h
Design hybrid retrieval with reranking for a multi-tenant help center, choose a configuration from measured quality and latency gates, and keep access rules intact through timeouts and model migrations.
AI Application Integration
4 h
Design a streaming interview-practice chat that keeps one answer correct across reconnects, several open tabs, and cancellation races, checks citations against current access, and never repeats its booking tool during recovery.
Multimodal & Document AI
4 h
Design a pipeline that turns a partly scanned course pack and a lesson recording into cited study material, tracking coverage, transforms, and cross-modal conflicts, and decide what it may answer while processing is still partial.
Model Serving & Inference
4 h
Choose how to serve interactive hints and offline drafts on four memory-limited accelerators: compute KV headroom, check measured tail-latency and quality gates, and design scheduling, overload handling, and verification.
Model Adaptation
4 h
Design a specialization program for a fictional hint model within GPU-hour and expert-review budgets: choose learning signals and parameterization, protect splits and evaluation, and plan a release with rollback. Then investigate a candidate that won a preference comparison while experts found wrong explanations.
Responsible AI & Governance
4 h
Decide what happens when a partner repurposes formative AI practice ratings for candidate screening, and design the authority, enforceable controls, evidence, and remedy that make the decision hold.
ML & Data Foundations
4 h
Design rolling-origin backtests, capacity-bound metrics, label-delay handling, and drift monitoring for a synthetic card-fraud model whose chargebacks arrive weeks late, then decide whether to promote, shadow, or hold a retrained candidate.
AI Developer Tooling
4 h
Design one agent harness for a fictional payroll company's developer machines and its CI agent, covering permissions, sandboxing, extensions, hooks, evaluation, and budgets. Then decide from predeclared rules which configuration rolls out first, where, and with which permissions.
Follow-up questions
An interviewer may ask these next.
- What does your system guarantee when a deletion races with an answer in flight?
- Where would you put a human approval step, and what exactly does it approve?
- What is the first alert you would set up?
- Which part would you simplify if you had half the time?
Scoring rubric
Interview ready at 8 of 10 with no 0.
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Problem framing | Jumps to a solution or solves the wrong problem | Restates the problem but misses key constraints or success criteria | Clarifies goals, constraints, and success criteria before solving |
| Correctness | Key claims or code are wrong | Mostly right with gaps or unhandled cases | Correct and precise, including edge cases |
| Evidence or testing | No way to verify the answer | Generic testing ideas | Concrete fixtures, metrics, or traces that would confirm or refute the approach |
| Trade-offs | One option, no alternatives | Names alternatives superficially | Compares options against the stated constraints and says what would change the decision |
| Communication | Disorganized or unclear | Understandable but meandering | Structured and concise; states assumptions; checks in with the interviewer |