Model Adaptation · 90 minutes
Decide whether to fine-tune
Decide whether a feedback assistant's failures call for prompting, retrieval, or fine-tuning, and explain the reviewed data, grouped split, and baseline comparison that fine-tuning would need.
Conceptual scenario
1.5 h
Used in interview rounds:
Technical Screen (Easy)1
Step 1: The question
The platform's team asks whether to fine-tune the model to fix these failures. Would you? Separate the failures by cause and name the first intervention for each. Then explain the training data, the split, and the evaluation you would need before trusting a fine-tuned model, and what must stay outside the model either way.
Scenario (fictional). Suppose a coding-practice platform runs a feedback assistant on a hosted model that its provider lets it fine-tune. For each help request, the assistant receives a one-line system prompt ("You are a friendly coding tutor."), the exercise, the learner's code, the failing test output, the requested help level (hint, explanation, or solution), and passages retrieved from the platform's documentation.
- Style guide. A hint-level reply opens with what works, names one issue, gives one hint without the solution, and stays under 80 words.
- Documentation change. Runtime Guide v3 replaced Runtime Guide v2 on 1 September 2026 and raised the time limit per test from 2 seconds to 5 seconds.
Three hint-level responses from last week
| Sample | Learner request | Response (shortened) |
|---|---|---|
| A | My pair-sum solution fails the empty-array test. Hint, please. | "Wrong. Check your loop." |
| B | My pair-sum solution fails the empty-array test. Hint, please. | A 260-word reply that praises the approach, re-explains the two-pointer technique, and ends with a corrected function. |
| C | Is my solution fast enough? The largest test takes 3.1 seconds. | "No. It exceeds the 2-second time limit (Runtime Guide v2, section 4). Try a faster approach." |
Review sample. A reviewer read 50 hint-level responses from last week: 18 broke the style guide (tone, length, or structure), 6 cited Runtime Guide v2, and 3 included a full solution. A response can appear in more than one count.
Available data. 1,400 past feedback responses on 120 exercises in 30 exercise families (variants of one problem). Between March and August 2026, expert reviewers rewrote 300 of them to match the style guide; the other 1,100 were never reviewed.
Constraints
- Use only the platform facts, samples, and figures shown; all of them are fictional
- Keep the answer to about three minutes spoken
Deliverables
- Each failure matched to a first intervention, with its cause
- The training data you would use for any fine-tuning, and what you would leave out
- How you would split train, validation, and test examples
- The baseline and held-out measurements that would show whether fine-tuning helped
- What stays outside the model, and what an adapted model costs to maintain
2
Step 2: Hints
Optional. Each hint gives a little more away.
3
Step 3: Your answer
Kept in this browser. Saving your score below keeps a copy with it.
0 / 5000
4
Step 4: Reference answer
5
Step 5: Score yourself
Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.
Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.
0 of 10 · Scored 0 of 5 dimensions
Ready at 8 of 10, with no 0 in Baseline and evaluation or Split integrity or Intervention per failure
Kept in this browser until you sign up, then it moves to your account.
Complete this level when you can match each failure in the samples to a first intervention with a reason, describe a reviewed dataset with a grouped split, and name the baseline and held-out measurements that would decide whether fine-tuning helped.
Your saved scores
Practice saves and mock interview answers on this question, newest first.
No saved scores yet. Score every dimension, then save.
6
Step 6: Follow-up questions
An interviewer may ask these next.
Is fine-tuning the same as adding documents to RAG?
Why can more training data hurt?
Does an adapter guarantee privacy?
Why keep a test set?
Can fine-tuning guarantee a schema?
7