Contact
What are you interested in?
Fine-Tuning.
Fine-tuning is the further training of an already pre-trained language model on your own examples, so that it reliably hits a particular behaviour, format or specialist vocabulary. It does not make the model cleverer or more current — new knowledge is the job of prompting and retrieval.
Fine-Tuning — Explained in Detail
In fine-tuning, a finished model is given additional example pairs of input and desired output, and its weights are adjusted slightly as a result. Retraining every parameter is expensive, which is why economical methods have become standard. With LoRA and related approaches the base model is left untouched and only small additional matrices are trained, switched in at inference time. That cuts compute and memory requirements substantially and makes it possible to run several specialised variants on the same base model without holding multiple copies of it.
Fine-tuning helps where a behaviour has to be hit repeatedly and exactly. Typical cases are a fixed output format, a particular tone of voice, classification into your own category system, or sector-specific phrasing a general model does not know. It does not help with currency or factual knowledge. Train a price list into a model and you will have to train it again at the next update, and you will still never be sure whether the model is reproducing the old version or the new one. For knowledge that changes, retrieval is the right answer, not training.
The effort sits in the data, not in the technology. Depending on the task, the order of magnitude is a few hundred to a few thousand clean examples, and quality beats quantity decisively: contradictory examples reliably teach the model to behave contradictorily. Every training run also needs a test set that is held out and never trained on, otherwise the improvement cannot be demonstrated. And a maintenance obligation is created: when the base model is retired, your variant has to be rebuilt and re-verified.
An everyday example: a business wants to sort incoming support messages into twelve in-house categories. The right sequence does not start with training but with a test set of two hundred messages sorted by hand. That is used first to measure how far a carefully written prompt with a few examples already gets. Only if that result falls short, and the errors show a pattern, does fine-tuning become worthwhile — and the same test set then proves whether it genuinely improved anything.
Related Page
RAG (Retrieval-Augmented Generation)Frequently Asked Questions About Fine-Tuning
As a rule of thumb: fine-tuning for behaviour, RAG for knowledge. If the model should answer in a fixed format, in a particular tone, or according to your own category system, training helps. If it should reproduce current facts from documents, prices or stock levels, retrieval helps, because the content can be changed without touching the model. Many applications combine the two.
For narrowly defined tasks such as classification or a fixed output format, a few hundred to a few thousand examples are often enough. What matters is not quantity but consistency: every example has to express the same rule, or the model learns the exception along with it. On top of that you need a separate test set of a few hundred cases that is never trained on, so the effect can be demonstrated at all.
Beyond the one-off training cost there are running costs. A fine-tuned model is often billed differently from the standard model, and it creates maintenance work. If the base model is retired or the task changes, you retrain and re-verify. That is why an honest comparison beforehand pays off: measure how far a good prompt with examples already gets you before committing to a training pipeline.
More Terms Starting with "F"
FCP (First Contentful Paint)
FCP measures how long it takes until the first visible content (text, image, SVG) appears in the bro...
First-Party Data
First-Party Data is data that a company collects directly from its own customers and website visitor...
Funnel (Marketing Funnel)
A Marketing Funnel visualizes the customer journey from first contact to purchase and beyond. It hel...
Ready for Your Project?
Apply this knowledge to your website — DLM Digital will help you.