Prompt, retrieve, or fine-tune? A decision tree that fits on a napkin

Start with the question, not the technique

Most of the confusion comes from treating these as competing camps. They're three tools for three different problems.

• The model doesn't know something. Your policies, your product catalog, last week's pricing. That's a knowledge problem.
• The model knows it but behaves badly. Wrong tone, wrong format, ignores instructions. That's a behaviour problem.
• It works, but it costs too much or runs too slowly. That's an efficiency problem.

The order we'd try things in

1. A better prompt and examples. Cheapest, fastest, easiest to undo. Give clear instructions, a few worked examples and a defined output format. Many "the model isn't good enough" problems end here.

2. Retrieval. If the answer depends on information that changes or is private, hand the model that information at question time instead of baking it in. You can update a document tomorrow. You can't update a trained model tomorrow.

3. Fine-tuning. Consider it when the task is stable, you have hundreds or thousands of good examples, and prompting is either too inconsistent or too expensive at your volume. It's good at teaching style and behaviour. It's a poor way to teach facts.

The signals that tell you to move on

• The same failure on the same kind of input after three prompt rewrites: look at retrieval or fine-tuning.
• Failures that cite stale or missing facts: retrieval.
• Correct answers in inconsistent format or tone at scale: consider fine-tuning.
• A large model doing a narrow job, and the bill is the problem: consider fine-tuning a smaller one.

What to do this month

1. Write 50 real test cases first. Without them you can't tell whether any change helped.
2. Fix the prompt before anything else. Time-box it to a day.
3. Add retrieval only for knowledge gaps. Look at what the model actually receives before you blame the model.
4. Hold fine-tuning until you can show the gap. "It would be nicer" isn't a reason.