Model Adaptation
Decide when to fine-tune and prove the change helped.
Choose between prompting, retrieval, and fine-tuning, validate a training dataset, and design an experiment that shows whether specialization helped.
3 questions
Questions
Easy
1.5 h
Decide whether a feedback assistant's failures call for prompting, retrieval, or fine-tuning, and explain the reviewed data, grouped split, and baseline comparison that fine-tuning would need.
Medium
3 h
Implement a split validator for a fine-tuning dataset, predict its results on fixture cases, find the leaks it cannot catch, and plan a supervised fine-tuning experiment against a prompted baseline.
Hard
4 h
Design a specialization program for a fictional hint model within GPU-hour and expert-review budgets: choose learning signals and parameterization, protect splits and evaluation, and plan a release with rollback. Then investigate a candidate that won a preference comparison while experts found wrong explanations.
What you will be able to do
- Distinguish prompting, retrieval, supervised fine-tuning, and preference learning by what each one changes
- Match an observed failure to the first intervention worth investigating, and say why
- Describe a trustworthy training dataset: reviewed targets, provenance, coverage, and privacy
- Split related examples so that evaluation measures generalization rather than recognition
- Explain why a lower training loss is insufficient evidence of product improvement, and what to measure instead
- Implement a dataset validator that enforces a stated contract in a fixed check order with exact error messages
- Group examples by exercise family and explain which leaks exact-match checks cannot detect
- Inspect chat-template rendering, loss masks, and truncation before training
- Design a blinded, paired comparison against a prompted baseline with a predefined decision rule and family-level uncertainty
Before you start
Suggested, not required.
Hands-on exercises
Interview rounds that use this topic
Topic boundaries
This topic covers choosing between prompting, retrieval, and fine-tuning, validating training data, and proving that specialization helped. Training infrastructure and distributed training are out of scope.
Related on CodeSnatch
System Design
Covers the model registry, lineage, offline versus online evaluation, and rollback that an adapted model's release relies on.
AI Engineering
Practice release gates with protected slices, paired comparisons, and stated uncertainty, the evidence that adaptation helped.
Practice problem
Hands-on check of split integrity for a chat fine-tuning dataset: unique IDs, family groups, and repeated conversations.
AI Engineering
Explains how retrieval brings evidence in at inference time, the main alternative to training facts into a model.
Practice problem
Practice scoring an evaluation set with separate denominators and per-slice results, the basis of a baseline comparison.
System Design
Covers data quality tests and lineage, habits a versioned training dataset needs as much as any warehouse table.
System Design
Its content dedup section teaches shingling and MinHash, a practical way to find near-duplicate training examples.
Behavioral Interview
Prepares stories on experiment rigor, catching your own analytical error, and data consent for ML interview loops.