The math behind AI, gently

What exactly is turning words and images into numbers? Let's try to understand. In a simple way. Here is the thing.

An AI model only ever does math. Adding, multiplying, nothing more. And math needs numbers. You cannot multiply the word cat directly.

An image is the easy case. It is already made of numbers. Every pixel is just a number for how bright it is. Zero is black, high is white.

Add color and each pixel becomes three numbers. Red, green, and blue. Line up the whole grid and the picture is one long list of numbers. Words are the hard case.

The letters c, a, t are not a quantity. There is no number hiding inside the word cat. So we hand it one. The trick is to give every word its own list of numbers.

Cat might become zero point two, minus one, zero point nine, and so on. That list is called a vector. Every word gets a list of the exact same length. Maybe three hundred numbers, maybe a few thousand.

The same size for all of them, so the math always lines up. And here is the payoff. A photo is a list of numbers. A word is a list of numbers.

Same shape, same rules. Now the model can finally do its math. So, to do anything, an AI turns everything into numbers. Images become a grid of pixel values.

Words become a vector, a fixed list. Then it is all just math. Quick check now. One question is coming up.

Let's see if it clicked.

How does a map of meaning work? Let's try to understand. In a simple way. Earlier, every word became a list of numbers.

Now read that list as coordinates. Like latitude and longitude, they drop the word onto a single point in space. This space has hundreds of directions, not just two. But the idea is the same.

Picture one huge map, with every word parked somewhere on it. Here is the key. Words with close meanings land close together. Cat and kitten sit almost on top of each other.

Airplane is on the far side of the map. Zoom out and neighborhoods appear. All the animals share one corner. Foods gather in another.

Colors form their own little cluster. Meaning has turned into geography. Why does this happen? During training, words that show up in similar sentences get nudged toward each other, over and over.

Related words drift together. Unrelated ones drift apart. So position now carries meaning. A tiny gap means two words are nearly the same.

A wide gap means unrelated. Nobody typed that in; it fell out of the training. Even directions mean something. The step from king to queen matches the step from man to woman.

The map lines these patterns up on its own. This is why search and recommendations work. Turn your question into a point, then grab its nearest neighbors. The closest dots are the most related answers.

So, numbers become coordinates. Similar meanings land nearby. And distance on the map becomes a stand in for meaning itself. Quick check now.

One question is coming up. Let's see if it clicked.

How does measuring similarity work? Let's try to understand. In a simple way. By now, every word and image is just a list of numbers.

So two ideas are two lists. And the question is simple. How close are these two lists? Here is the trick.

Treat each list as an arrow, pointing out into space. Similar meanings point the same way. Different meanings point apart. To measure that, we use one calculation.

Walk down both lists together. Multiply each matching pair. Then add up all those products. That single total is called the dot product.

A big positive total means the two arrows line up. A total near zero means they point in unrelated directions. That is the whole signal. Really, this number is tracking one thing.

The angle between the two arrows. A small angle means very similar. A wide angle means very different. But there is a catch.

A longer list gives a bigger total, even when the direction is the same. So raw size can fool us. The fix is to divide out the length of each arrow. Now only the direction counts, not the size.

This cleaned up version is called cosine similarity. And it lands on a tidy scale. Plus one for the same direction. Zero for unrelated.

And minus one for exact opposites. So, to compare two ideas, turn them into arrows, and check the angle between them. One clean number, from plus one down to minus one. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is embedding an idea? Let's try to understand. In a simple way. In the last steps, every word got a number.

But that number was just a name tag. It says which word, and nothing about its meaning. To embed something means to place it inside a space. So we take each idea and drop it at one spot.

One idea becomes one point. And a point is just its coordinates. So the idea turns into a short list of numbers. Every list has the same length.

Maybe three hundred numbers long. Here is the real shift. Unlike the plain name tag, these numbers carry meaning. Loosely, each slot leans toward some quality of the idea.

Think of a map. A city gets a latitude and a longitude. Two numbers that fix where it sits. An embedding does the same, with hundreds of numbers instead of two.

And nobody types these coordinates in by hand. They are learned during training. The model nudges every point until the whole layout becomes useful. In practice it is a big table.

One row for every word in the vocabulary. To embed a word, the model grabs its row. That row is the embedding. So the payoff is this.

Meaning turns into position. Once an idea is a point, the model can measure it, add it, compare it. Just arithmetic. So, to embed an idea is to give it a spot in a space of numbers.

A fixed list of learned coordinates. A name tag that finally means something. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a matrix? Let's try to understand. In a simple way. By now, a piece of data is just a list of numbers.

A vector. But we rarely leave it alone. We want to transform it into a new, more useful list. So meet the matrix.

A matrix is simply a grid of numbers, laid out in rows and columns. That rectangle of numbers is the whole thing. Its shape is just how many rows and columns it holds. Say four rows across three columns.

Think of each row as its own little list. Now the action. Take one row of the grid, and lay it against your input list. Multiply each matching pair, then add them up.

That gives one number. Then do the same with the next row. Every row produces one number. Stack those results top to bottom, and you have a fresh new list.

Notice what happened to the size. Three numbers went in. Four came out, one per row. So the grid can grow or shrink your data.

And each output is a blend of every input, mixed by the numbers in that row. So the matrix recombines your data into new features. This single move, a grid times a list, is the workhorse of a model. It runs millions of times, pushing the data forward, step after step.

So, a matrix is a grid of numbers. Multiply it by a list, one row at a time, and out comes a transformed list. Data in, new data out. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is a matrix transformation? Let's try to understand. In a simple way. Start with one idea, drawn as an arrow from the center.

Multiply it by a matrix, and out comes a new arrow. Its tip has moved somewhere new. And it does this to every point at once. Picture the whole grid of space getting picked up and reshaped.

One rule, the same for all. How do we know where things land? The columns of the matrix hold the answer. They mark where the two unit arrows go.

Everything else follows. The first basic move is a stretch. Numbers on the diagonal pull the grid longer or wider. Above one stretches things out.

Between zero and one squeezes them in. The second basic move is a rotation. Every arrow swings around the center by the same angle. The lengths stay the same, only the directions turn.

A real matrix mixes both at once. Some stretch, some turn, in a single multiply. Almost any reshaping is just stretch and rotate, combined. But one rule always holds.

Straight grid lines stay straight and evenly spaced, and the center never moves. That restraint is what makes it a linear map. This is exactly what a layer does. It applies a learned matrix to every embedding, stretching and rotating the meaning-space, so the ideas it cares about line up.

So, a matrix is an action, not just a grid. It stretches and rotates every arrow at once, reshaping a whole space of ideas with one clean rule. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a layer? Let's try to understand. In a simple way. By now, data is just a list of numbers.

A layer is one step. It takes that list in, and hands back a new, more useful list. At its heart sits a matrix, a grid of weights. Multiply the input list by it, and every output becomes a weighted blend of all the inputs.

But a layer adds one more thing. A bias. A small list with one number per output, added on top. It nudges each result up or down.

Then comes the twist. An activation function bends each number. A common one is simple. Keep it if it is positive, and flatten it to zero if not.

Why bend at all? Without it, stacking layers would just collapse into one flat matrix. The bend is what lets the network curve, and draw real decision lines. So, multiply, add the bias, then bend.

That three step move, all together, is one single layer. A list goes in, a richer list comes out. Think of each output as a tiny question asked of the input. Is this royal?

Is this loud? One layer turns raw numbers into a set of answers. Now stack many layers, each feeding the next. Early ones catch simple patterns, later ones combine them into ideas.

That depth is why we call it deep learning. So, one layer is a matrix, plus a bias, plus a bend. It reshapes your list of numbers into sharper features, one clean step of thinking. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is a model's knowledge? Let's try to understand. In a simple way. Here is the surprise.

An AI has no database of facts to look up. It does not search for answers. There is no filing cabinet inside. All it has is numbers.

Its knowledge lives entirely in a huge pile of numbers called weights. Nothing else is stored. Where do these weights sit? Inside every layer, filling the grids we multiply by.

Stack all those grids together, and that is the whole model. Count every one of those numbers, and you get the model's parameter count. A small model has billions. A large one has far more.

But here is the strange part. No single number holds a fact. The idea of Paris is smeared across thousands of weights at once. Knowledge is spread out, never filed in one spot.

So where did these numbers come from? Training wrote them. Before training they are random noise. Training slowly tunes each one until the whole pile encodes what the model learned.

Once training ends, the numbers freeze. The very same weights answer every question you ask. The model never looks anything up. It just runs your input through those fixed numbers.

This is why an open model is just a file of weights you download. That file of numbers is the knowledge. Copy it, and you have copied everything the model knows. So, a model's knowledge is not a database.

It is millions or billions of weights, learned in training, frozen afterward, and spread across every layer. Quick check now. One question is coming up. Let's see if it clicked.

How does an AI know when it is wrong? Let's try to understand. In a simple way. Every time a model makes a prediction, there is also a correct answer it was aiming for.

To improve, it first has to notice the gap between the two. So the model does something simple. It measures how far its guess is from the truth. That distance becomes a single number.

That number has a name. The loss. A big loss means the guess was badly off. A loss near zero means it was almost perfect.

But there is a catch. If you just subtract, a guess that is too high looks negative, and one too low looks positive. Being off in either direction should count as wrong. So the common fix is to square the difference.

Squaring throws away the sign, and it punishes big misses far more than small ones. Now the loss behaves. It is zero only when the guess exactly matches the answer, and it grows the further off the model is. One example is not enough.

The model checks thousands of them and averages all their losses into one overall score. That average is what it truly wants to shrink. This whole recipe, guess, compare, turn the gap into a number, is called the loss function. It is the model's only sense of being wrong.

So an AI does not feel wrong. It computes a number, the loss, that grows with every mistake. Shrinking that number is the entire goal of learning. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the slope of our mistake? Let's try to understand. In a simple way. The model already has a loss, one number for how wrong it is.

It wants that smaller. But each weight can move up or down. Which way helps? So it runs a tiny experiment.

Nudge one weight up a hair, and watch the loss. Does the mistake grow, or shrink? That answer is a slope. Picture the loss as a hill, with the weight as your spot.

The slope is how the ground tilts where you stand. Put plainly, it is how much the loss changes for a tiny change in the weight. A little rise over a little run. The sign points the way.

A positive slope means the loss climbs as the weight climbs. So to go downhill, move the weight the other way. The size matters too. A steep slope means this weight has a big effect, and the bottom is far.

Near zero, the ground is flat and you are close. But a model is not one weight. It is millions, each with its own slope. Gather them all into the gradient.

It points where the loss rises fastest, so downhill is the opposite. And it never tests each weight by hand. Calculus hands back every slope at once, in a single backward sweep through the network. So the slope of our mistake is the tilt of the loss under one weight.

Gather every tilt into the gradient, and the model knows which way is downhill. Quick check now. One question is coming up. Let's see if it clicked.

How does a single learning step work? Let's try to understand. In a simple way. Picture the loss as a valley.

High up on the walls, the model is very wrong. Down at the bottom, it is right. Learning is just walking toward that bottom. We already know the slope under our feet.

It points uphill, toward more loss. So to get better, we step the opposite way, straight downhill. Here is the actual move. Take the current weight, and subtract the slope from it.

That one subtraction shifts the weight in the downhill direction. But we do not subtract the whole slope. We keep only a small slice of it, so each move is one gentle step. That small slice has a name.

The learning rate. It is a dial that sets how big each step is. Usually a tiny number, far below one. Why subtract, and not add?

Subtracting moves us against the slope, which is the downhill way. Adding would climb the wall and make the loss worse. So after this single step, the weight has shifted a little, and the loss is a little smaller. The model is now slightly less wrong than before.

One step barely moves us. So we repeat it, again and again, each time reading the new slope and taking another small step downhill. This patient loop is called gradient descent. So, to get smarter, a model reads the slope, flips its direction, and steps a little way downhill.

Repeat that many times, and the loss slowly shrinks. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a step that's too big? Let's try to understand. In a simple way. Remember the learning rate.

It is the dial that sets how big each downhill step is. Turn it up, and the steps grow. Turn it down, and the steps shrink. Set it too small, and every step is a timid crawl.

The model does inch toward the bottom, but it could take forever to get there. Set it just right, and each step lands closer to the bottom than the last. The model glides down in a smooth, steady descent. Now set it too big.

One giant leap flies clear over the bottom, and lands high up on the far wall. Higher than where it started. The next leap is just as wild. It shoots back across, and climbs even higher.

The model bounces from wall to wall, in a widening zig-zag. So the loss grows instead of shrinking. The model is getting worse, not better. When training runs away like this, we say it has diverged.

The slope only knows the ground right under your feet. It does not know how far away the bottom is. A giant leap trusts that one hint far too much. The fix is simple.

Shrink the learning rate until the bouncing stops. The best choice is the largest step that still settles calmly toward the bottom. So, too small and learning crawls. Too big and it bounces and blows up.

The learning rate lives in a sweet spot between the two. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is softmax? Let's try to understand. In a simple way. At its final step, a model gives out one raw score for every option it is weighing.

One number for cat, one for dog, one for fox. These raw numbers are called logits. But these scores are messy. Some are negative, some are large, and together they add up to nothing in particular.

You cannot read them as confidence. What we really want is probabilities. Each option between zero and one, and all of them together adding up to exactly one. Then the numbers mean something.

So softmax works in two moves. First, it raises a fixed number, e, about two point seven, to the power of each score. Every result is now positive. Second, it adds up all those positive results, and divides each one by that total.

Now every value sits between zero and one, and they add up to one. That two step recipe is softmax. It turns the raw scores into a clean set of probabilities, one confidence value for every option. Why the exponent, instead of just dividing the raw scores?

Because raising to a power stretches the gaps. The top score pulls far ahead, so the winner looks confident. And notice, softmax never reshuffles the ranking. The highest score always becomes the highest probability.

It only reshapes the numbers, it does not change who wins. So softmax is the translator. Raw scores go in, and out come probabilities that are all positive and sum to one, ready to read as confidence. Quick check now.

One question is coming up. Let's see if it clicked.

How does an AI weigh every possible answer? Let's try to understand. In a simple way. When an AI picks the next word, it does not settle on just one.

Behind the scenes, it weighs every word it knows, all at once. Its whole vocabulary is on the table. Tens of thousands of possible words, and each one gets a score for how well it fits right here. Those scores are turned into probabilities.

Every option now holds a share of the model's confidence, a number like zero point six, or zero point zero one. And here is the key. All of those probabilities add up to exactly one. Together, they form a single distribution over every answer.

Sort them from high to low and you get a ranked list. A few strong favorites sit at the very top, holding most of the confidence. Below the favorites stretches a long tail. Thousands of words, each with a tiny sliver of probability, almost zero but never quite gone.

The shape of this spread tells you a lot. One tall spike means the model is sure. Many bars of similar height mean it is torn. This full distribution is the real output.

It has not chosen yet. Picking one word from the spread is a separate step that comes after. So the AI never hands you one answer. It weighs them all, scores every option, and shows a whole distribution of possibilities.

Quick check now. One question is coming up. Let's see if it clicked.

How does an AI choose its next word with a bit of randomness? Let's try to understand. In a simple way. By now, the model has scored every word into a probability.

One word leads, the others trail. But it can write only one. So how does it choose? The safe move is to always grab the single highest word, every time.

This is greedy picking. Reliable, but robotic. The same prompt gives the same answer. So instead, the model rolls a weighted die.

Each word's chance of winning equals its probability. The leader still wins most often, but the runners up now get a real turn. This is sampling. Run it twice, and you can get two different words.

That little bit of chance is what makes writing feel alive and creative, instead of stiff and predictable. How much randomness? One dial sets it, called temperature. It reshapes the probabilities before the roll.

Turn it down, and the leader towers over the rest. Turn it up, and the field flattens. Low temperature stays safe and focused, almost always taking the favorite. High temperature spreads the odds wide, so bold, unusual words get a real shot.

Too high, and it blurts nonsense. To stay safe, models often trim the tail first. Keep the top few words, or the smallest group covering most of the weight. Then sample from just those.

Creative, never absurd. So picking a word is its own step. Sample from the spread for variety, and let temperature dial the boldness up or down. Quick check now.

One question is coming up. Let's see if it clicked.