Fine-tuning vs RAG for business software

Fine-tuning changes the AI model itself by training it further on your examples. RAG leaves the model untouched and supplies the relevant information from your own sources with every question. For business software that works with current facts, such as customers, contracts, orders and invoices, RAG or a direct database connection is almost always the right choice. Fine-tuning is useful for form and behaviour, not for knowledge that changes.

What is fine-tuning?

A language model has been trained on an enormous amount of general text. With fine-tuning you train it once more, on a much smaller set of your own examples. The model adjusts part of its weights in the process. Afterwards it behaves differently: it writes in a particular style, follows a fixed format or recognises a specific type of document better.

What fine-tuning does well:

• Enforcing a fixed form. A model that always returns a finding in exactly the same structure, with the same fields.
• Learning jargon and classification. A model that has to sort your engineers' job sheets into the categories you use internally.
• Making a smaller model perform like a larger one. For a narrow, repeated task a small fine-tuned model may be enough, which makes inference cheaper and faster. See inference explained.

What fine-tuning does badly:

• Remembering facts that change. A model trained on your customer list from January does not know in June that customers have joined or left.
• Exact recall. A model that saw a contract during training cannot reliably quote the indexation percentage. It has learned patterns, not stored a table.
• Showing where something comes from. A fine-tuned model cannot point to a source. The answer comes from its weights.

What is RAG?

RAG stands for retrieval-augmented generation. The model stays as it is. With every question, the software first looks up the relevant information and puts it in the context window, together with the question. The model writes its answer based on what it has just received.

The lookup can work in two ways. For text, such as contracts and manuals, usually with embeddings: the passages closest in meaning to the question. How that works is covered in embeddings explained for business software. For structured data, such as orders and invoices, an ordinary database query is better. The difference between the two is covered in RAG vs database queries for business data.

What RAG does well:

• Staying current. If a contract changes, the next question finds the new version. Nothing needs retraining.
• Showing sources. The system knows which passages it supplied, so it can say where a statement comes from.
• Respecting permissions. What a user may not see is not retrieved. With fine-tuning, everything is in the model, for everyone who uses it.

What RAG does badly:

• Finding the wrong passage. If the lookup goes wrong, the model writes a convincing answer based on the wrong source.
• Handling large volumes. RAG supplies a selection. Questions about everything at once, such as a total over a year, belong not in RAG but in a query. A model is not a calculator for thousands of lines.

Fine-tuning and RAG side by side

Why business data does not belong inside a model

There is a reason that weighs more heavily than the technology: control over your data.

If you use customer data, contract amounts or personal data to fine-tune a model, that data sits inside the model in a diffuse way. You cannot point to where. You cannot remove it selectively when a customer asks for erasure. The model may reproduce fragments of it to someone who should not have seen them. Under data protection law such as the GDPR, that is an awkward position. More on this in how to protect personal data in AI systems.

With RAG and direct queries, the data stays in your own systems. The model only sees it at the moment of the question, and only as far as permissions allow. Delete a record in the source and it does not come back.

That fits a principle that applies to every form of revenue control: your own systems remain the source of truth. The AI reads, compares and explains. It does not itself become the place where the figures live. How that division of roles works out in a complete system is covered in how AI works within Revenue Intelligence.

When does fine-tuning make sense?

Fine-tuning is not a bad idea; it is just often used for the wrong purpose. It suits tasks where the behaviour is fixed and the facts come from outside:

• A model that converts invoice lines from suppliers' PDFs into a fixed format. The form is always the same; the content comes from the document itself.
• A model that sorts free-text notes into a fixed set of categories, such as cancellation reason or type of additional work.
• A model that always phrases findings in the style and structure your organisation is used to.

Even then, the facts the model works on come in through RAG or a query. Fine-tuning and RAG are not mutually exclusive. A fine-tuned model can work perfectly well with retrieved data.

Worked example: what happens if you train prices into a model

Worked example: suppose a business fine-tunes a model on its price agreements in January, so that an assistant can say which price a customer should pay. There are 250 customers with their own price agreement.

• On 1 April, 90 contracts are indexed.
• In May, 15 customers join and 8 leave.
• In June, an employee asks the assistant which price customer X should pay. The model gives January's price, with conviction.

To prevent that, the model would have to be retrained after every change. That is expensive, slow and still gives no guarantee that the exact amount comes back correctly. With RAG or a query, the assistant retrieves the current price from the system where it lives and states where it came from. Anyone invoicing at outdated prices has exactly the leak described in revenue leakage from wrong prices.

Fine-tuning or RAG? A decision guide

Ask yourself these questions for every AI application on business data:

1. Does the information change? Yes: RAG or query. No, and it is about form or behaviour: fine-tuning is an option.
2. Does the user need to see the source? Yes: RAG or query.
3. Are some users not allowed to see everything? Then the data does not belong in the model.
4. Does it contain personal data? Then the data does not belong in the model.
5. Is it about counts, totals or amounts? Then the arithmetic belongs in a query, not in a model.
6. Is the task narrow, repeated and fixed in form? Then fine-tuning can help, especially to lower the cost per call.

Frequently asked questions

Is RAG always better than fine-tuning?

For questions about facts that change, yes. For a fixed form, style or a narrow classification task, fine-tuning can be better. Many systems combine the two.

Can a vendor use my data to fine-tune its model?

Only if the contract allows it. Ask explicitly, and have it excluded if you do not want it. This is about training, not about the ordinary use of the model when a question is asked.

Is a fine-tuned model safer because the data is not sent with every question?

No. The data is then inside the model itself, where you cannot remove it selectively and where per-user permissions do not work. For sensitive business data that is less controllable, not more.

What does fine-tuning cost?

That depends on the model, the provider and the amount of training data. There is no fixed figure. Do allow for recurring costs if the task changes, because every adjustment requires a new training run.