When fine-tuning is worth it

Fine-tuning is training a model a little further on your own data so it gets better at your specific task. It is the most glamorous technique in local AI and the most overused. Most people who fine-tune would have been better served by prompting, RAG or a different model.

The honest ranking

Before fine-tuning, exhaust these first:

1. A better prompt. A surprising amount of "the model is bad" is "the instructions are bad."
2. A better model. Switching to a stronger or better-suited model fixes more than most fine-tunes.
3. RAG. If your problem is "the model does not know my data," retrieval beats training. Training does not reliably store facts; it nudges style and behaviour.

When fine-tuning actually pays

• A consistent style or format you want in every answer, without spelling it out each time.
• A niche behaviour the model does not have (a tone, a structured output shape, a domain convention).
• Teaching a small model a specific task so it can run cheaply, instead of renting a big model.

The tell: fine-tuning changes how the model answers, not what it knows. If you need it to know your documents, that is RAG. If you need it to behave a certain way, that is fine-tuning.

The real cost

Fine-tuning needs good labelled data, which is the expensive part. If you do not have a few hundred clean examples, you are not ready to fine-tune, and the result will not generalise. People underestimate the data and overestimate the technique.

The decision rule

Fine-tune only when: you have tried prompt, model and RAG, the gap is behavioural (not factual), and you have the labelled data to show it. Otherwise, spend the effort on data and evaluation instead, which pay off no matter what you do next.

The method track is the discipline behind this: measure first, then decide what to change.