Keep the trace

The final piece of the method is the least glamorous and the most valuable: keep the record of everything you tried and what happened. This trace is what separates expertise from experience, and it is what you will actually use months from now.

What the trace contains

For every experiment:

what you changed and why,
the exact setup (dataset, model, quant, settings, prompt),
the scores and the failures,
one line of judgement: ship it, keep looking, or revert.

A changelog, a spreadsheet, a folder of dated results. The format does not matter; the existence and the habit do.

Why it pays off

You stop relearning. Six months later, when the same problem comes up, you have the answer instead of a vague memory.
You can explain yourself. "Why did we choose this model?" becomes a lookup, not a story.
You build on your own work. Each experiment becomes a brick, not a one-off.

What people actually do

The setups and people who are good at this all keep a trace, in one form or another: a repo of eval results, a changelog next to the pipeline, a document of decisions. The ones who do not keep re-fighting the same battle.

The habit

Make recording part of the loop, not an afterthought. The run is not done until the result is written down. If it is too much friction, simplify the format until it is not.

The bigger picture

The trace is your memory, externalised. Combined with the replay loop, it is the whole method in one image: a growing table of experiments, each one a fact, each one built on the last. That is what "expertise by the numbers" actually means, and it is the thing no amount of enthusiasm can fake.

Keep the trace. Future you will read it and be glad.