The fine-tuning workflow

If you have decided fine-tuning is the right move, the workflow is mostly data work, and only a little training. Get the data right and the training is almost boring; get it wrong and no tool saves you.

The steps

1. Define the behaviour. One concrete change you want, in writing. "Answer like a compliance reviewer" is a goal; "always start with the rule number and a one-line verdict" is a spec.
2. Collect examples. Gather real input/output pairs that show the behaviour. Aim for a few hundred clean examples, not ten.
3. Clean and split. Remove junk, dedupe, and split into a train set and a hold-out test set. The test set is never used in training; it is how you will know if you succeeded.
4. Format the data. Most tools expect a specific format (instruction/response pairs). Follow it exactly.
5. Train. Start with a parameter-efficient method like LoRA or QLoRA. It trains a small adapter on top of the frozen model, so it is cheap and runs on a single card.
6. Evaluate. Run the hold-out set and compare before/after on your actual task, not just the loss number.

The tools

• Unsloth — the popular choice for fast LoRA fine-tuning on modest hardware.
• Axolotl — a flexible, config-driven trainer.
• MLX — Apple Silicon fine-tuning. The tool matters less than the data. Pick one and learn it well.

The mindset

Fine-tuning is an experiment, not a purchase. You will not get it right the first time. The workflow above, especially the hold-out set, is what lets you iterate: change the data, retrain, re-evaluate, keep the better one.

The honest warning

A fine-tuned small model rarely beats a bigger untuned model on raw intelligence. It beats the same model at your task. Judge it on that, not on a leaderboard.

The method track's evaluation lessons are the natural next step: this workflow only works if the "evaluate" step is real.