Prompt, RAG or fine-tune?

Think of the fixes as a ladder. Climb only as high as you need.

1. Prompt and context. Minutes to set up. Edit the prompt to change it.
2. RAG. Days to weeks. You build the search, the data flow and the permissions.
3. Fine-tuning. Weeks. You need examples, training and tests.
4. Training a model from scratch. Months of work and a big budget. It is almost never right for using AI at work.

A long text sent with every request costs money each time. Prompt caching can help.

Think of a new worker. A clear briefing is prompting. A card for the company library is RAG. A long training course is fine-tuning. It changes their habits for good. But you would not use a course to teach this week's prices. You would hand them the price list.

Fine-tuning keeps costing. You collect and check good examples. You pay for training. You test the result on a fixed set of cases, called evals. Then you do it all again when you move to a newer base model.

Some API providers let you fine-tune some of their hosted models. With an open-weight model, you can do it on your own computers.

A fine-tuned model can still mix up the facts it saw, and state them with confidence. There is no reliable way to make it forget one record. Fine-tuning can also weaken the model's other skills and some of its safety training. So test it widely, not only on the task you trained.

Training all of a big model's weights needs a lot of memory. It also makes a full-size copy for every version. A smarter way freezes the original weights. It trains only a small add-on, called an adapter. This is called LoRA, short for Low-Rank Adaptation.

The adapter is a small file you can swap in and out. So one base model can serve many tasks, with one adapter each. LoRA is still fine-tuning, so the rule stays the same: habits, not facts. QLoRA combines LoRA with quantization.

A big model can also teach a small one. The big "teacher" writes many example answers. The small "student" is fine-tuned on them. This is called distillation. The student is cheaper and faster to run. It keeps a good part of the teacher's skill on those tasks, but not all of it.

For example, DeepSeek made small "R1-Distill" models from answers by its big DeepSeek-R1 model. Before you learn from any model's answers, read its terms. Some providers forbid using their outputs to train competing models.

The handbook plan became RAG. The handbook changes every quarter, and staff want to see which paragraph an answer came from. It uses the search from Search by meaning: embeddings and RAG. Who may see which document is in Permission-aware RAG.

Months later, IT used LoRA to fine-tune a small open-weight model. Its one job: sort 50,000 helpdesk tickets a week into 40 groups. Two years of already sorted tickets were the examples. That job needs speed, low cost and exact labels, not new knowledge.