Interview Practice
Debugging & Evaluation
Locate a failure stage by stage and prove the fix with evidence.
What interviewers evaluate
- Stage-by-stage hypotheses
- A confirming check per hypothesis
- A minimal fix and a regression test
- No weakened controls
What you will be asked
Diagnose a failure stage by stage and prove the fix.
In the room
You get logs, fixtures, or a small dataset with a wrong result, and you produce a diagnosis table, a minimal fix, and a regression check.
Timebox
45 minutes
Mock interview
A timed answer to a random question from this level, then the reference answer and your score.
How to practice
- Use the linked questions: write one hypothesis per stage, the check that would confirm it, and the result, before you reveal the reference answer.
- Give each hypothesis a few minutes and move on when the evidence rules it out.
- With a peer: they answer only yes or no to each check you propose.
What a strong answer shows
- One hypothesis per stage, each with a confirming check
- A minimal fix that keeps authorization and validation in place
- Metrics with explicit denominators
- A regression test or fixture for the failure
Questions for this level
Each question opens on its Practice tab, and Previous and Next follow this list.
LLM Foundations
3 h
Work out why a model's answer looks cut off or incomplete by reading termination metadata, token budgets, and stream evidence before changing any setting.
RAG & Data Pipelines
3 h
Trace a missing policy exception through ingestion, eligibility, and context packing without weakening access rules.
Evaluation & Observability
3 h
Score a six-case evaluation set with a deterministic harness, report success, coverage, and answered accuracy with named denominators, and keep missing outputs visible in the totals.
Model Serving & Inference
3 h
Summarize per-request timing records into latency, TTFT, request TPOT, and throughput with failures kept in the counts, then use a KV memory estimate and stage timings to find what limits a long-prompt load profile.
ML & Data Foundations
3 h
Implement an audit that flags leaky features, preprocessing, time splits, selection, and mismatched reports, predict its findings on five fixture experiments, and diagnose why a churn model that looked excellent offline failed after launch.
Follow-up questions
An interviewer may ask these next.
- Your fix improved the average but one slice got worse. What do you do?
- How many cases do you need before you trust a two-point improvement?
- What does a missing termination reason tell you?
Scoring rubric
Interview ready at 8 of 10 with no 0.
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Problem framing | Jumps to a solution or solves the wrong problem | Restates the problem but misses key constraints or success criteria | Clarifies goals, constraints, and success criteria before solving |
| Correctness | Key claims or code are wrong | Mostly right with gaps or unhandled cases | Correct and precise, including edge cases |
| Evidence or testing | No way to verify the answer | Generic testing ideas | Concrete fixtures, metrics, or traces that would confirm or refute the approach |
| Trade-offs | One option, no alternatives | Names alternatives superficially | Compares options against the stated constraints and says what would change the decision |
| Communication | Disorganized or unclear | Understandable but meandering | Structured and concise; states assumptions; checks in with the interviewer |