Fine-tuning in practice

How does nudging a model's internal weights work? Let's try to understand. In a simple way. A trained model is really just a giant pile of numbers.

We call those numbers its weights. Every fact and skill the model has learned lives inside them. Fine tuning does not start from nothing. It begins with a model that was already trained on a huge pile of text.

Training a model from scratch means starting from random numbers and feeding it a mountain of data. That takes months and costs a fortune. Fine tuning skips all of that. You show the model a small set of your own examples, and gently nudge its existing weights.

The nudge uses the same learning it always did. The model guesses, sees how wrong it was, and each weight slides a little toward a better answer. The key word is gently. Most of the weights barely move at all.

The model keeps almost everything it already knew. And because you started from a model that is already clever, just a handful of examples is enough to lean it toward your task. What you get out is the same model, with its knowledge quietly reshaped to match your style, or to handle your specific job. So, fine tuning is not building a new brain.

It is taking a trained one and nudging its weights just enough. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is prompting versus fine-tuning? Let's try to understand. In a simple way. There are two very different ways to change how a model behaves.

One is prompting. The other is fine-tuning. They pull completely different levers. Prompting means writing your instructions, and maybe a few examples, inside the request.

The skill is already in the model. Your words steer which one comes out. But that prompt only lives in that one request. The weights never change.

Start a fresh chat, and it remembers nothing. Prompting steers, it does not teach. Fine-tuning is the other lever. You show the model hundreds of examples and nudge its internal weights.

The new behavior gets baked in, and stays, no prompt required. So when do you just prompt? When the skill is already inside, and only needs coaxing. A tone, a format, a clear instruction.

A sharper prompt unlocks it. And when do you fine-tune? When the model simply cannot do it, even with your best prompt. That is the signal to teach a genuinely new skill.

Each lever has a price. Prompting is instant, cheap, and easy to change. Fine-tuning costs data, compute, and time, but the skill is baked in. So here is the rule of thumb.

Try prompting first, because it is faster and far cheaper. Reach for fine-tuning only when a good prompt keeps falling short. So, prompting steers skills the model already has, and fades after each request. Fine-tuning bakes a new skill into the weights, for good.

Prompt to steer, fine-tune to teach. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a library versus a lesson? Let's try to understand. In a simple way. You have a model and want it better at your task.

There are two different ways to help it, and people mix them up. The first way, give it a library. A shelf of documents it can look things up in. The model never changes; you hand it the right pages when asked.

This has a name. Retrieval. At question time, you search the shelf and paste the matching passages into the prompt. The facts live outside the model.

The second way, give it a lesson. You sit the model down and teach it, with many examples that nudge its weights. That is fine tuning, baking the skill in. So the difference is where knowledge lives.

A library keeps it outside, looked up fresh each time. A lesson bakes it inside the weights, always on. When should you give a library? For facts.

Things that change, private data, many specifics. Update the shelf, and the model is instantly current. When should you give a lesson? For a skill.

A new tone, a strict output format, a way of reasoning. Behavior you want by heart, not looked up. Here is the classic mistake. People fine tune to cram in facts.

But weights are a leaky memory. Facts blur, go stale, and updating means retraining. They belong in the library. So, a library for what it should know.

A lesson for how it should act. Match the tool to the need. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a good example dataset? Let's try to understand. In a simple way. A fine-tune does one thing.

It copies the examples you hand it. So your dataset is the entire lesson. Whatever pattern lives inside, the model imitates. Which means every label has to be right.

If an answer in your data is sloppy or wrong, the model learns to be sloppy and wrong too. Garbage in, garbage out. So quality matters far more than raw size. A small set of clean, careful examples beats a huge pile of noisy ones.

Every bad example teaches a bad habit. Next, coverage. Your examples should span the real range you expect. Short ones and long ones, easy ones and rare edge cases.

Whatever you leave out, the model may fumble. They also have to agree. If two examples handle the same situation in opposite ways, you send a mixed signal. Pick one style, and hold it across the set.

And watch for accidental patterns. Say every positive example is short, and every negative one is long. The model may quietly learn length, not meaning. It grabs any shortcut you leave lying around.

Your data should also look like real use. Train on the messy inputs you will actually see, not tidy textbook ones. Otherwise it shines on your examples and stumbles in the wild. So, a good dataset is the lesson itself.

Clean labels, wide coverage, one steady style, and inputs that match real life. Craft the examples with care, because the model becomes them. Quick check now. One question is coming up.

Let's see if it clicked.

How does a prompt and completion pair work? Let's try to understand. In a simple way. When you fine-tune a model, the training data is not loose text.

It comes in pairs. Each example has two halves. A prompt, and a completion. The prompt is the input, the kind of message a user would send.

The completion is the exact answer you want back. One worked example. During training, the model reads the prompt as context, then it is graded only on the completion. So the completion side is what it actually learns to write.

Now the key rule. Every pair should share the same shape. The same wording around it, the same separators, the same layout. Consistency is what the model locks onto.

And here is the trap. The format you train with must match the format you use later, when the model is running. Train one way, ask another, and it gets confused. Keep the completion clean.

It holds only the output you want, nothing extra. No, sure, here you go. And give it a clear stopping point, so the model knows where the answer ends. Chat models dress this up with roles.

A system message, a user message, an assistant message. Same idea though. The user turn is your prompt, the assistant turn is your completion. So, each training example is a pair.

A prompt going in, a completion you want back. Keep every pair in the same shape, and match it to how you will use the model. Quick check now. One question is coming up.

Let's see if it clicked.

How does fine tuning work with so few examples? Let's try to understand. In a simple way. You might expect that teaching a model needs a mountain of data.

Millions of examples. So it feels wrong that a few hundred could ever be enough. But here is the trick. You are not starting from zero.

The model already spent months reading the whole internet. It arrives fluent. So fine tuning is not teaching from scratch. It is a nudge.

You are steering a skill the model already has toward the exact shape you want. How few is few? Often just fifty to a few hundred good examples. Sometimes a couple dozen is enough to lock in a tone or a format.

With so few, every example carries real weight. The model sees the pattern repeat, and it generalizes the rule, rather than memorizing each line. This is why quality beats quantity. Fifty clean, consistent examples will out-teach five thousand messy ones.

One contradictory example can confuse the whole batch. And more is not always better. Past a point, doubling your examples barely moves the needle. The first hundred teach the pattern; the next thousand mostly repeat it.

One limit though. A few examples can reshape behavior, a style or a habit. They cannot pour in a whole new field of facts. That needs a different tool.

So, a small, clean set of examples is powerful because the model is already smart. You are nudging, not building. Steer with a few, and it follows. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is one training epoch? Let's try to understand. In a simple way. When you fine-tune a model, you hand it a pile of training examples.

One epoch is one full pass through all of them. Every example, seen once. But the examples are not fed in one at a time. They are gathered into small groups called batches.

It works through them one batch at a time. Each batch triggers one learning step. The model makes predictions, checks how wrong they are, then nudges its weights a little toward the right answer. Inside that step, four things happen.

A forward pass to predict, a loss score for the error, gradients pointing downhill, and a weight update. That is one step. So how many steps are in one epoch? The number of examples divided by the batch size.

A thousand examples in batches of ten is a hundred steps. One more detail. Before each epoch, the examples are shuffled into a new order. That keeps the model from leaning on the sequence instead of the content.

By the end, every example has been seen exactly once. But the weights were nudged once per batch, so many small steps have added up. And the epoch is the repeat unit. Usually you run several, passing over the same data again and again, each pass shaping the weights a bit more.

So, one epoch is one full sweep through your data, split into batches. Each batch is a step that nudges the weights, and the epoch is what you repeat. Quick check now. One question is coming up.

Let's see if it clicked.

How does the learning rate work? Let's try to understand. In a simple way. During fine-tuning, the model nudges its weights to fix its mistakes.

The learning rate is the number that sets how big each nudge is. Picture the model walking downhill into a valley. The bottom is where it fits the data best. The learning rate is its stride, how far it moves on each step.

Set it too high and the strides are huge. The model leaps past the bottom and overshoots. The loss bounces around, and can even blow up. Set it too low and the model creeps.

Each step barely moves the weights. It gets there eventually, but training takes far longer than you can afford. So you want a middle value. Big enough to make progress, small enough to settle gently at the bottom.

That is a good learning rate. Here is the fine-tuning twist. Your starting model is already skilled. You want to adjust it, not rebuild it.

So you use a much smaller rate than training from scratch. Push the rate too high on an already good model, and the big steps trample what it knew. It can lose skills it used to have. Small steps protect them.

In practice the rate is not fixed. It often starts tiny, rises to a peak, then slowly decays. Bold moves early, careful nudges near the end. So, the learning rate is your step size.

Too big overshoots, too small crawls. For fine-tuning, keep it small, refining the model without breaking it. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is memorizing instead of learning? Let's try to understand. In a simple way. You fine-tune a model to teach it a skill.

You show it examples, hoping it picks up the pattern behind them, so it can handle new inputs. But there are two ways it can fit your examples. It can learn the general pattern. Or it can simply memorize the exact answers, one by one.

Memorizing is like a student who crams the answer key instead of understanding the subject. They ace the practice test, then freeze on the real exam. The telltale sign is a gap. The training loss keeps dropping, looking great, but the score on fresh, unseen examples starts getting worse.

A memorized model parrots its training answers word for word. Change the question a little, and it breaks, or repeats an answer that no longer fits. Why does this happen? Too many passes over too few examples.

The model sees the same handful so often that it just stores them, exactly. The fix starts with a held-out set. Set some examples aside and never train on them. Then watch how the model does on those.

Stop the moment that held-out score stops improving, even while the training loss keeps falling. That is early stopping. And give it more varied examples. It is far harder to memorize a thousand different cases than ten repeated ones.

Variety forces the pattern out. So, memorizing means storing answers, not learning the skill. Watch the held-out gap, stop early, and feed it variety. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is full fine-tuning? Let's try to understand. In a simple way. You begin with a model that is already trained.

Inside it sit billions of numbers, called weights. They hold everything it has learned so far. Full fine-tuning makes one bold choice. Every single one of those weights is allowed to change.

Not a few. All of them. Nothing is held fixed. So you show it your own examples.

The model guesses, you measure how wrong it is, and that error reaches back into every weight at once. Each weight then gets nudged a tiny bit, toward the setting that would have made the answer better. Every knob turns, just a little. Repeat this across your whole dataset, and the model slowly shifts.

Because nothing was frozen, it can reshape any part of its behavior. It is the deepest change you can make. But touching every weight is expensive. To update each one, training must also store a gradient and an optimizer value beside it.

That is several times the model's size in memory, so it wants big graphics cards. And when you finish, you own a brand new copy of the model. Just as large as the original, with every weight rewritten for your task. It is powerful, but heavy.

Reach for it when you have real data, strong machines, and need deep change. Otherwise, lighter methods freeze most of the model. So, full fine-tuning unfreezes every weight and adjusts them all. The most complete change you can make, and the most costly.

Quick check now. One question is coming up. Let's see if it clicked.

How does freezing a model and adding small adapters work? Let's try to understand. In a simple way. Full fine-tuning changes every weight in the model.

For a big model, that is billions of numbers to update, and a fresh copy to store. Expensive, and slow. So here is a cheaper idea. Freeze the whole model.

Lock every original weight in place, so not a single one can change. Then slip in a few tiny new layers, called adapters. They sit beside the frozen ones, adding just a sprinkle of fresh weights. Now you train.

But the gradients only flow into the adapters. The frozen giant stays exactly as it was. Only the small pieces learn. The most popular version is called LoRA, short for low-rank adaptation.

Instead of one huge update matrix, it learns two skinny ones, whose product stands in for that update. How small is small? Often under one percent of the weights actually train. So the memory and the compute drop way down.

And because the base never changes, each adapter is a tiny file. Snap one on for legal, another for code. One frozen model, many skills. The surprise is the quality.

For most tasks, these little adapters match full fine-tuning, at a small fraction of the cost. You barely give anything up. So, freeze the big model, add small adapters, and train only those. Same power, far less to move.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is catastrophic forgetting? Let's try to understand. In a simple way. Start with a model that already does many things.

It can chat, translate, write code, answer trivia. All of that lives in one big shared set of weights. Now you fine-tune it on one narrow job. Say, only customer support replies.

Every training example points in that single direction. But here is the catch. Those replies are learned by nudging the very same shared weights. Nothing in the new data reminds the model of its old skills.

So step by step, the weights drift toward the new task. And the settings that once held translation, or coding, slowly get written over. This is called catastrophic forgetting. The model grows sharp at the new job, while quietly losing things it used to do well.

Why so drastic? A neural network has no separate drawer for each skill. They all overlap in the same numbers, so overwriting one can dent another. The main fix is to remind it.

Blend some general examples back into the training data, so the old skills keep getting practiced. This is called rehearsal. You can also train gently. A small learning rate, fewer passes, or freezing most of the model, so the original weights barely move.

So, fine-tuning on narrow data can overwrite the shared weights that held broad skills. Push too hard on one thing, and the model forgets the rest. Quick check now. One question is coming up.

Let's see if it clicked.

How does measuring a model's improvement work? Let's try to understand. In a simple way. You are fine-tuning a model, and it is running.

But is it actually getting better, or just spinning? You cannot feel it. You need a number. That number is the loss.

After each batch, the model reports how wrong its guesses were. A high loss means far off. As training works, it should drop. But the training loss alone can fool you.

The model is being graded on the very examples it studies. It could simply memorize them, and still look great. So you hold some examples back. A validation set, kept aside, never shown during training.

You grade the model on these, and they reveal what it truly learned. Now watch the validation loss across each epoch. If it keeps falling, the model is genuinely improving on data it has never seen. That is real progress.

Plot both together. Training loss slides down and down. But when the validation loss flattens, or starts creeping up, more training is no longer helping. That turning point is your signal to stop.

Save a checkpoint at the lowest validation loss, and keep that one. It is the best version, before things drift. One caution. A low loss is not the whole story.

Also check a task score, like accuracy, and read a few answers. Loss can look fine while replies still miss. So, to know if your model is improving, watch the loss on data it never trained on. The curve tells you when to stop.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is your starting model? Let's try to understand. In a simple way. When you fine-tune, you do not start from a blank model.

You start from one already pretrained on mountains of text. So the model you pick is your starting point. Fine-tuning moves it only a short way from there, not from nothing. That means the starting point sets your ceiling.

A weak base, given a little data, stays weak. A strong base has far more to build on. Picture it as distance to travel. If the base already sits close to your task, the gap is short.

Less data, less training, better results. So match the base to your job. Fine-tuning for code? Begin with a model already fluent in code.

For a new language, start from one that has seen it. Size matters too. A bigger base holds more knowledge, but costs more to train and to run. Pick the smallest one that clears your bar.

One more fork. A raw base model just predicts text. An instruct model already follows directions. For a chat task, that head start saves you a lot.

And you inherit what the base brings. Its tokenizer, its context length, its license, even its blind spots. You build on that foundation. So, fine-tuning starts from a pretrained model and only nudges it.

Pick a base that is capable, close to your task, and the right size. Quick check now. One question is coming up. Let's see if it clicked.

How does serving a fine-tuned model work? Let's try to understand. In a simple way. Training is over.

You now have a checkpoint, a saved set of weights that know your task. But a file on a disk cannot answer anyone. Serving means putting that checkpoint to work. You load the weights onto a machine and let it answer real requests, one after another.

This is inference, not training. No gradients, no optimizer. The weights are frozen. Text comes in, the model runs one forward pass, and tokens come out.

The model sits behind an endpoint. Your app sends a prompt, the model runs, and the answer comes back. If you trained a small adapter, you serve the base plus that adapter. One base can hold many adapters, swapped to switch tasks.

The weights must fit in a graphics card's memory. Inference needs less memory than training did, but the model still must stay loaded and ready. Many users may call at once. To use the card well, requests are grouped into batches.

More at a time raises throughput, but watch each person's delay. The cost also changes shape. Training was one big bill. Serving charges you while it runs, by the hour, or by the tokens it makes.

Keep the model behind a stable address. Then you can slot in a better checkpoint later without touching the app, and watch real traffic. So, serving turns your finished weights into a live service. Load them, answer requests, and pay for the time it runs.

Quick check now. One question is coming up. Let's see if it clicked.