LLM Foundations · 3 to 4 hours
Select a model under quality, cost, and latency limits
Recommend a model or routing policy from measured quality, cost, and latency, say what evidence is still missing, and revise the decision when the traffic mix changes.
4 h
Used in interview rounds:
Technical Screen (Hard)Overview
Select and defend a model configuration under quality, latency, context, and cost constraints. The goal is a decision with measurable acceptance criteria, stated uncertainty, and a validation plan. GPU serving, retrieval architecture, and rollout have their own topics.
Study time: 3 to 4 hours
Interview format: System design discussion with calculations
By the end you can
- Turn a product task into mandatory gates and preferences with an explicit success definition
- Compute weighted quality and token cost per request and per month
- Explain why a point estimate at a threshold is not a reliable pass
- Evaluate a routing policy, including router errors and combined latency
- Revise a decision when the workload mix changes, and record what would reverse it
Prerequisites
Helpful background: Comfort with percentiles, weighted averages, and experimental design helps.
In this guide
- Translate a product task into constraints
- Compare capabilities on representative work
- Context capacity versus effective use
- Quality and uncertainty
- Latency and token economics
- Routing and fallback trade-offs
- Selection scenario
- Work the scenario at today's traffic mix
- Evaluation and decision record
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.
Related on CodeSnatch
Lessons, problems, and articles that build the skills this question uses.
AI Engineering
Turns slices, calibrated graders, and stated uncertainty into a ship decision, extending this question's evidence work.
Practice problem
Practice summarizing a load test into throughput, nearest-rank p95 latency, and time to first token.
AI Engineering
Applies the same gates-then-cost reasoning to hosting a model yourself, with utilization and breakeven volume.
Practice problem
Practice cost per success with failed attempts included, so a cheaper token price is judged by finished work.
Behavioral Interview
Shows how to state a call with calibrated confidence and name the signal that would prove it wrong.
Practice the interview question
Answer it in your own words before you look at the reference answer.