Prompting, RAG and fine-tuning: what changes where?

What is the difference between prompting, RAG and fine-tuning? Each one changes a different thing. Prompting changes the instructions you send with every call; the model itself does not change, so a fix is instant and costs a few tokens. Retrieval-augmented generation (RAG) fetches relevant documents at question time and puts them in the prompt, so the model can use facts it was never trained on, including ones that changed last week. Fine-tuning changes the model's own weights by training it on examples. It shifts behaviour, style and format, and it comes along with every call, but it is slow and costly to redo, and it is an unreliable way to add facts. Use prompting when the shape or the task is wrong, RAG when facts are missing or keep changing, and fine-tuning when behaviour or style must change. This film runs one task through all three so you can see which fault each one fixes.

The task is one support request, recorded on 11 September 2026 with google/gemini-3.8-flash at temperature 0, five runs per condition. Priya paid for Tallyboard's annual plan on 1 August and asks for a refund on 10 September, 40 days later. The reply has three separate jobs. It must be a JSON object that software can act on, which is about shape. It must apply the refund policy updated on 1 September, 45 days for annual plans, which is a fact the model cannot know. And it must sound like the support team, which is behaviour. The bare model fails all three in every run. It writes friendly prose, makes up a 30-day refund window, and signs off in its own voice.

Each lane is a request plate, a model box and a reply easel, with three check chips that turn green when a check passes in all five runs. Lane 1 adds one line of instructions asking for the JSON object. It costs 39 prompt tokens, and all five replies are valid JSON, but all five still refuse on 30 days. Lane 2 leaves the instructions alone and inserts the policy passage, as a retriever would. All five replies quote 45 days and grant the refund, but they are back in prose and in a generic voice. Lane 3 opens the model. No model was trained for this film. The recorded lane is a proxy: forty past replies in the house style are placed in the prompt, which gets the behaviour a fine-tune on those replies would aim for. All five replies then open with "Thanks for getting in touch, Priya." and close with the team's sign-off.

The scoreboard reads as a diagonal: each fix repairs its own column and none of the others. That is the practical test. If the output has the wrong shape, change the instructions. If it has a wrong or missing fact, put the fact in the context. If it has the wrong manner, and more instructions have stopped helping, that is when training earns its cost. Put all three into one call and every check passes in all five runs.

The failure mode is using fine-tuning to add facts. The past replies were written before the policy changed, so the refund examples say 30 days. Lane 3 learns the style and the stale rule together, and it declines Priya in perfect house style in all five runs. A fine-tuned model has the same problem in a harder place: the old rule would sit in its weights, where you cannot read it or edit it without training again. Published results point the same way: Ovadia et al. (2023) found retrieval beat unsupervised fine-tuning at getting new facts into a model, and Gekhman et al. (2024) found that fine-tuning on facts a model did not already know was slow to learn and made it hallucinate more once learned. In the all-three call, the retrieved passage outranks the stale examples and the reply quotes 45 days. That is the arrangement to aim for: behaviour in the weights, facts retrieved fresh on every call.

Cost is the other half of the choice. The format line and the passage cost tens of tokens. The forty examples cost 2,685 extra prompt tokens on every call, and at list prices a thousand calls go from $0.71 for the bare request to $3.40. Training moves that cost out of each request. A LoRA adapter (Hu et al., 2021) of rank 8 on the attention projections of Llama 2 7B (Touvron et al., 2023) trains 8,388,608 of its 6,738,415,616 weights, 0.12 percent, and leaves the rest frozen. The film does not show a training curve or a training bill, because none was measured.

The honest caveats. Lane 3 is an in-context proxy, not a fine-tune, and a trained adapter could learn more or less of the style. There are five runs per condition of one model on one request, all near identical at temperature 0, so this shows the mechanism rather than measuring rates. The policy passage was inserted directly rather than found by a retriever, so retrieval quality, which is where most RAG systems fail, is not tested here. The checks are strict and simple: JSON must parse with nothing around it, the policy check looks for 45 days and no 30, and the style check reads the opening and the sign-off.