Decide whether to fine-tune

Model adaptation changes learned behavior using additional training. Full fine-tuning updates the model's weights broadly. Parameter-efficient methods train a smaller set of parameters, such as adapters. Prompting supplies instructions at inference time without updating weights. Retrieval supplies external evidence at inference time.

These techniques can be combined. Suppose a coding-practice platform retrieves the current grading rubric, uses a fine-tuned model to deliver consistent feedback, and validates the model's JSON with application code. No single technique replaces the others' responsibilities.

Fine-tuning may teach information, but it is usually a poor sole mechanism for frequently changing facts, exact citations, or tenant-specific access rules. Diagnose retrieval failures before training the generator to compensate for missing evidence.

Supervised fine-tuning, or SFT, uses examples of desired outputs conditioned on inputs. During training, a language model is encouraged to assign higher probability to the target response tokens.

An example is a demonstration, not merely a topic label. If all feedback examples praise incorrect code, training can reinforce that behavior. If targets always include the full solution, the model may reveal answers even when a learner requests a hint.

For a coding-practice feedback assistant, a useful example contains the exercise, the learner's submission, the rubric or other relevant evidence, the requested help level, and an expert-reviewed response. Include only context available at inference. A target relying on hidden reviewer notes cannot be reproduced reliably when those notes disappear.

This is an illustrative record, not a universal provider upload schema:

{
  "id": "feedback-001",
  "group_id": "exercise-family-17",
  "messages": [
    {"role": "system", "content": "Give one useful hint without the final solution."},
    {"role": "user", "content": "My loop misses the last array element. What should I inspect?"},
    {"role": "assistant", "content": "Compare your loop's stopping condition with the index of the final element."}
  ]
}

Roles, special tokens, and chat templates affect the token sequence the model actually sees. Training and inference must use compatible formatting. Inspect several rendered examples before training. Do not assume that a valid JSON record creates a correct training sequence.

Prefer reviewed, representative demonstrations over indiscriminate volume. Cover common inputs, ambiguous cases, mistakes, legitimate refusals, and edge cases. Include difficult examples only when their targets are reliable.

Track provenance, permitted use, reviewer decisions, dataset version, and the intended skill. Remove credentials and unnecessary personal information. A private dataset does not guarantee that a trained model cannot reproduce sensitive text.

Synthetic examples can expand coverage, but repeated model mistakes can become training targets. Review samples and keep evaluation independent of the generation process. Many paraphrases of the same exercise do not provide many independent tests of generalization.

Training examples update parameters. Validation examples guide model selection and training decisions. A held-out test set estimates performance after those choices are complete.

Split related examples together. Put variants of one exercise, source document, or conversation into the same partition. Otherwise a model can appear to generalize while recognizing a lightly edited training example. For time-sensitive behavior, a temporal holdout may better represent future traffic.

Do not repeatedly tune against the final test set. Once it influences decisions, it has effectively become a development set; establish a new independent evaluation when needed.

A model can lower its loss by learning predictable formatting while remaining wrong on important reasoning cases. Evaluate the actual behavior you care about.

Full fine-tuning changes many or all model parameters. Adapters add a smaller trainable component to a frozen base. LoRA represents selected weight updates with low-rank matrices. Its central benefit is reducing trainable parameter and optimizer-state requirements; the frozen model still needs memory and computation.

An adapter depends on a compatible base model and configuration. Treat the base revision, tokenizer, chat template, and adapter as one release bundle. An adapter file alone does not fully specify the deployed model.

Preference data compares responses to the same input. Reviewers choose which response better satisfies a defined rubric. The goal might be better correctness, more useful hints, or less unnecessary verbosity.

A preferred response is not automatically a flawless response. If both candidates are wrong, record that issue rather than silently turning the less wrong output into a gold demonstration. Keep criteria explicit so a preference for polished prose does not conceal incorrect code.

Compare against the strongest practical prompting baseline using the same held-out inputs and comparable generation settings. Evaluate correctness, instruction adherence, structured-output validity, unsupported claims, and appropriate refusal behavior. Include regression cases from unrelated but important capabilities.

Measure cost and latency under the actual serving configuration. Training expense is only part of the lifecycle cost; deployment, maintenance, new data review, and regression testing also matter.

Use the table in "Choose the intervention from the failure" and the advice in "Evaluate the decision".

This continues the feedback assistant from the classification exercise.