Interview Practice
Live Coding
Implement a small, testable piece of an AI application against fixtures.
What interviewers evaluate
- Clarifying the contract
- Writing or running fixture tests
- Handling edge cases
- Narrating failure modes
What you will be asked
Implement a small, testable piece of an AI feature against fixtures.
In the room
A contract, a fixture table, and a function to implement in a shared editor. The interviewer watches how you clarify, test, and handle edge cases.
Timebox
45-60 minutes
Mock interview
A timed answer to a random question from this level, then the reference answer and your score.
How to practice
- Open a linked question, read only the context and the fixture table, then solve the exercise in the coding workspace before you look at the reference code.
- Run the tests early and often, and say out loud what each failure tells you.
- With a peer: they answer contract questions only when asked and note every assumption you make silently.
What a strong answer shows
- Clarifies the input contract and the order of checks
- Writes or runs fixture tests before calling the work done
- Handles the fixture edge cases and at least one more
- Takes trusted values (tenant, user, permissions) from context, never from untrusted input
- Names the remaining failure modes
Questions for this level
Each question opens on its Practice tab, and Previous and Next follow this list.
Prompting & Structured Output
3 h
Implement a dependency-free validator for a ticket contract, classify each fixture failure by layer, and design a two-attempt recovery that never commits invalid output.
Embeddings & Retrieval
3 h
Rank four help articles by shared words and by cosine similarity, score both rankings against labels, and explain a dense miss and an accidental lexical success.
RAG & Data Pipelines
3 h
Trace a missing policy exception through ingestion, eligibility, and context packing without weakening access rules.
Tool Calling & Agents
3 h
Implement a get_ticket adapter that validates model-proposed arguments, takes tenant scope from trusted context, and passes a fixture table, then explain how a bounded orchestrator handles timeouts, invalid calls, and parallel lookups.
Security & Privacy
3 h
Implement a context selector that enforces tenant, group, and deletion rules before ranking, then design tests proving that forbidden records and internal notes never reach the model-bound payload.
Model Adaptation
3 h
Implement a split validator for a fine-tuning dataset, predict its results on fixture cases, find the leaks it cannot catch, and plan a supervised fine-tuning experiment against a prompted baseline.
Multimodal & Document AI
3 h
Implement a validator for page-region citations and predict its result on fixed cases. Then design extraction for a mixed PDF with a rotated scan and a table split across two pages.
AI Application Integration
3 h
Implement a pure reducer that keeps a streamed answer consistent through stale runs, replays, gaps, and stops, and predict its transitions. Then review a fetch integration for decoding, framing, cancellation, and end-of-stream defects.
Model Serving & Inference
3 h
Summarize per-request timing records into latency, TTFT, request TPOT, and throughput with failures kept in the counts, then use a KV memory estimate and stage timings to find what limits a long-prompt load profile.
AI Product & Experimentation
3 h
Write the experiment contract and sample-size plan for a staged-hints A/B test, implement a validated per-arm summary of outcomes and costs, and explain which checks and guardrails must pass before a lift can justify shipping.
ML & Data Foundations
3 h
Implement an audit that flags leaky features, preprocessing, time splits, selection, and mismatched reports, predict its findings on five fixture experiments, and diagnose why a churn model that looked excellent offline failed after launch.
AI Developer Tooling
3 h
Implement the request handler of a small read-only MCP server that validates tool arguments from their declared schemas and puts each failure in the right channel, then review the trust model of adding a shell tool and installing a third-party server.
Follow-up questions
An interviewer may ask these next.
- Which test would catch a check done in the wrong order?
- What changes if this function becomes asynchronous?
- How would you make this operation safe to retry?
- What would you log, and what must you never log?
Scoring rubric
Interview ready at 8 of 10 with no 0.
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Problem framing | Jumps to a solution or solves the wrong problem | Restates the problem but misses key constraints or success criteria | Clarifies goals, constraints, and success criteria before solving |
| Correctness | Key claims or code are wrong | Mostly right with gaps or unhandled cases | Correct and precise, including edge cases |
| Evidence or testing | No way to verify the answer | Generic testing ideas | Concrete fixtures, metrics, or traces that would confirm or refute the approach |
| Trade-offs | One option, no alternatives | Names alternatives superficially | Compares options against the stated constraints and says what would change the decision |
| Communication | Disorganized or unclear | Understandable but meandering | Structured and concise; states assumptions; checks in with the interviewer |