Model Adaptation · 3 to 4 hours
Design a model specialization program
Design a specialization program for a fictional hint model within GPU-hour and expert-review budgets: choose learning signals and parameterization, protect splits and evaluation, and plan a release with rollback. Then investigate a candidate that won a preference comparison while experts found wrong explanations.
System design and experimental reasoning
4 h
Used in interview rounds:
AI System Design (Hard)1
Step 1: The question
Using the specialization scenario, design this quarter's specialization program for the hint model. Choose learning signals and parameterization within the GPU-hour and expert-review budgets, and explain splits, data lineage, evaluation gates per difficulty level, and rollout under the hint-only rules. Then use last quarter's failed candidate to show how your program would diagnose it and what evidence would change your decisions.
Constraints
- The quarter has 400 GPU-hours, evaluation included, and 3,600 expert reviews
- Every hint-only rule is mandatory at every difficulty level
- Exercises change weekly, and partner exercises and learner logs without opt-in must not train shared weights
- The current prompt-plus-retrieval system is the baseline every candidate must beat
Deliverables
- Learning signals, and the evidence that would justify each training stage
- Parameterization with its arithmetic, and a GPU-hour plan that includes seeds and evaluation
- An allocation of the expert reviews, plus splits, lineage, and evaluation gates per level
- A diagnosis of last quarter's candidate and what you would do with it
- Release manifest, rollout, rollback, and data-removal handling
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.