Interview Practice
Debugging & Evaluation
Locate a failure stage by stage and prove the fix with evidence.
What interviewers evaluate
- Stage-by-stage hypotheses
- A confirming check per hypothesis
- A minimal fix and a regression test
- No weakened controls
What you will be asked
Read a trace or an incident and say which stage failed.
In the room
The interviewer shows a short trace (retrieval results, the context sent, and the answer) or an incident timeline, and asks where the failure happened and what you would check next.
Timebox
45 minutes
Mock interview
A timed answer to a random question from this level, then the reference answer and your score.
How to practice
- For each linked question, read the trace or incident first and write your stage diagnosis before you read the guide.
- Name the evidence for each claim: which line of the trace shows it.
- With a peer: they reveal one extra piece of evidence only when you ask for it.
What a strong answer shows
- Separates retrieval, context packing, and generation
- Points at the specific evidence behind each conclusion
- Proposes one next check rather than a broad rewrite
- Never weakens a control, such as an access filter or a limit, to make a symptom go away
Questions for this level
Each question opens on its Practice tab, and Previous and Next follow this list.
Evaluation & Observability
1.5 h
Use a request trace to tell retrieval, context-packing, and generation failures apart, and report answer quality with a named denominator next to operational signals.
RAG & Data Pipelines
1.5 h
Explain how documents become evidence for a generated answer, and why an answer can be wrong even when it cites a real document.
Production & Reliability
1.5 h
Explain why a slow model provider raises concurrency and retry load, using latency, throughput, rate limits, deadlines, and a bounded response strategy.
Model Serving & Inference
1.5 h
Split a slow first token into queue wait, prefill, and emit from a server trace, and choose a serving configuration per workload instead of by throughput alone.
Follow-up questions
An interviewer may ask these next.
- The right document was retrieved but not cited. Which stage failed?
- What would you add to the trace to make this faster next time?
- How do you tell a slow dependency from a retry storm?
Scoring rubric
Interview ready at 8 of 10 with no 0.
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Problem framing | Jumps to a solution or solves the wrong problem | Restates the problem but misses key constraints or success criteria | Clarifies goals, constraints, and success criteria before solving |
| Correctness | Key claims or code are wrong | Mostly right with gaps or unhandled cases | Correct and precise, including edge cases |
| Evidence or testing | No way to verify the answer | Generic testing ideas | Concrete fixtures, metrics, or traces that would confirm or refute the approach |
| Trade-offs | One option, no alternatives | Names alternatives superficially | Compares options against the stated constraints and says what would change the decision |
| Communication | Disorganized or unclear | Understandable but meandering | Structured and concise; states assumptions; checks in with the interviewer |