Chips: the silicon behind AI

How does a chip turn electricity into logic? Let's try to understand. In a simple way. Deep inside a chip, the only thing flowing is electricity.

A voltage on a wire. Either high, or low. That is all the chip ever has to work with. So we pick a code.

High voltage means one. Low voltage means zero. Every wire now carries a single bit. One tiny yes or no.

To do anything with those bits, we need a switch. That switch is called a transistor. A tiny gate that either lets current pass, or blocks it. Here is the clever part.

A transistor is controlled by electricity itself. One voltage flips the switch on, another flips it off. Electricity controlling electricity. Now wire a few transistors together in a careful pattern.

You get a logic gate. It reads the bits coming in, and outputs a fresh bit. One pattern gives you NOT. One bit in, flip it.

Another gives AND. Output a one only when both inputs are one. A handful of gates is all you need. Chain those gates together, and they can add two numbers.

Stack millions of them, and they can carry out any calculation at all. All of this is etched into a sliver of silicon. Billions of transistors, each far thinner than a human hair, switching billions of times a second. So a chip is really just switches.

Voltage becomes bits, transistors become gates, and gates become logic. Electricity, turned into thinking. Quick check now. One question is coming up.

Let's see if it clicked.

How does an AI model work? Let's try to understand. In a simple way. Inside a model, there are no words, and no ideas.

Every input becomes numbers. And the model itself is just a giant pile of numbers, called weights. So what does the model do with them? Take each input number, multiply it by a weight, and add all the results together.

One number comes out. That is the whole trick. Multiply, then add. Multiply, then add.

Engineers call it a multiply and accumulate. It is the single most common step in all of AI. A single layer does thousands of these weighted sums at once. Line them all up, and the whole thing is exactly what mathematicians call a matrix multiplication.

And a model does not have one layer. It stacks hundreds of them. Each layer is more grids of numbers, and more multiply and add. How much math is that?

To handle a single word, a large model runs billions of these multiply and add steps. A full answer climbs into the trillions. So there is no hidden reasoning circuit inside. Every answer you ever see is just this one arithmetic operation, repeated at a colossal scale.

And this is the key. The math is simple, but the amount of it is staggering. So a chip that does multiply and add really fast is exactly what AI needs. So an AI model is just numbers, and one repeated step.

Multiply and add, done billions of times. A giant math problem, and nothing more. Quick check now. One question is coming up.

Let's see if it clicked.

How does a CPU solve problems in sequence? Let's try to understand. In a simple way. A program is really just a list of instructions.

Add these two numbers. Store the result. Compare it to something. It is an ordered recipe, step after step.

The CPU is the worker that reads that recipe. And here is the key. It does one instruction at a time, strictly in order. For each instruction it runs the same little loop.

Fetch the instruction from memory. Decode what it means. Then execute it. Fetch, decode, execute.

How does it know which instruction is next? A program counter. It simply holds the address of the next step, and ticks forward by one each time. Now the trick.

A clock inside the chip ticks billions of times a second. So the CPU races through that sequence, billions of steps every second. But a recipe is not always a straight line. Sometimes it hits a decision.

If this, do that. The counter jumps to a different instruction, and the sequence carries on. This is what a CPU is built for. A few very powerful cores, each one designed to charge through a single chain of steps as fast as possible.

But there is a catch. It works one lane at a time. Give it a huge pile of identical sums, and they all wait in that single line. So a CPU is a fast, sequential worker.

Fetch, decode, execute, again and again, one step after the next. Brilliant at following a recipe in order. Quick check now. One question is coming up.

Let's see if it clicked.

Why does a GPU attack problems in parallel? Let's try to understand. In a simple way. The math inside an AI model is not one clever calculation.

It is the same tiny sum, a multiply and an add, done over and over on different numbers. And here is the key. None of these sums waits on another. Each one can be worked out entirely on its own.

A CPU has a few brilliant workers, but it takes these sums single file. One finishes, the next begins. With millions to do, that line gets long. A GPU makes the opposite bet.

Instead of a few fast workers, it fields a huge crowd of simple ones, set loose at the same time. There is a trick that makes this cheap. Every worker runs the exact same instruction, just on its own slice. One order, shouted to the whole crowd.

Now, any single worker is slower than the CPU's star. But that is not the game. Thousands finish together. You win on total work, not lap time.

But this bet only pays off in one case. The work must be independent. If step two needs step one's answer, the crowd cannot help. You are back in a line.

And this is exactly why GPUs and AI fit together. A model is just those independent multiply and adds, piled high. Perfect work for a crowd. So a GPU wins by width, not speed.

Split the work into independent pieces, hand them to a crowd, and finish them all at once. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a core in a GPU? Let's try to understand. In a simple way. Start with one core.

Think of it as a tiny calculator. It grabs two numbers, multiplies them, and hands back the result. A CPU, a normal processor, has only a few cores, but each is large and powerful. A GPU makes the opposite trade.

Many cores, each one small. How do you make a core small? You strip it down. No clever tricks, no big private memory.

Just the raw math, and little else. Because each core is so tiny, thousands fit on one chip. A modern GPU can carry more than ten thousand cores. The cores are not loose.

They sit in teams. Each team shares one controller that hands the same instruction to every core. Here is the catch. Every core in a team runs the same step at the same moment.

Same instruction, each on its own data. This fits one kind of work perfectly. Apply the same simple math to a huge grid of numbers. That is the matrix math inside AI.

But if the cores need different things, they cannot. Some sit and wait while the others go. Lockstep is a gift only when the work is identical. And one core alone is slow.

The magic is the crowd. Thousands of slow cores, finishing together, move a mountain of numbers each second. So, a GPU core is a stripped down calculator. Tiny, so thousands fit.

Bunched into teams that march in lockstep on the same math. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the memory wall? Let's try to understand. In a simple way. Start with the good news.

A modern chip has thousands of cores, and they crunch numbers incredibly fast. Raw math is not the problem. But the numbers those cores work on do not live inside the cores. They sit in a separate pool of memory, off to the side.

So every number has to make a trip. From that memory pool, across the chip, to a hungry core waiting to use it. And here is the trouble. The road between them is narrow.

Only so many numbers can travel across it each second. So the cores finish their work and then just wait. Fast calculators, sitting idle, because the data has not arrived yet. This gap has a name.

The memory wall. Compute speed raced ahead, while the speed of fetching data crawled behind. Picture a wide factory fed by one thin pipe. The factory is not slow.

The pipe is. That backup is the traffic jam. AI makes it far worse. A large model has billions of weights, and every single one must be streamed in for each step.

So the real limit is bandwidth, how fast you can feed the cores. Not how fast they can multiply. The chip is starved, not slow. So, the memory wall is simple.

The cores are quick, but the road to their data is narrow. Feeding them is the bottleneck. Quick check now. One question is coming up.

Let's see if it clicked.

How does stacked memory work? Let's try to understand. In a simple way. A chip is packed with cores that crunch numbers at blistering speed.

But a core can only work on numbers that reach it. When they arrive too slowly, all that speed just waits. Ordinary memory sits far across the board, reached by one narrow path. Numbers crawl along it almost single file.

So the hungry cores are left waiting for data. The first fix is distance. Put the memory right beside the processor, on the very same package. Now every trip is short.

The second fix is shape. Instead of spreading the memory chips out flat, we stack them into a tiny tower, several dies tall. Then we drill wires straight down through the whole tower. Thousands of vertical links, one chip on top of another.

Engineers call them through silicon vias. A short trip, and thousands of wires together, make one very wide road. Numbers travel side by side, not single file. That is the fast lane.

The payoff is huge. Several terabytes of data can pour into the cores every second. This stacked, close, wide memory even has a name. High bandwidth memory.

And it matters for every word a model writes. Each new word must pull the whole model down from memory. So this wide lane sets the real speed. So, stack the chips into a tower, wire them wide with thousands of lanes, and keep them close.

That is how you build a fast lane. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a chip built just for AI? Let's try to understand. In a simple way. A normal chip, a CPU or a GPU, is a jack of all trades.

It runs anything. A game, a spreadsheet, a browser. That flexibility is not free. Most of the chip is control and memory, deciding what to do next.

Only a sliver does the math. But an AI model does one thing, over and over. Multiply numbers, then add them up. The same simple math, billions of times a second.

So what if you build a chip that does only that? Throw out the flexibility. Spend every transistor on multiply and add. A chip wired for a single job has a name.

An ASIC. It cannot run a game. It does one task, beautifully. Google built exactly such a chip.

The TPU, or tensor processing unit. Its whole body is a dense grid of multiply and add units. Why Google? AI was running inside every product.

Search, translate, photos. Billions of times a day. General chips could not keep up. The reward is efficiency.

For the same electricity, a specialized chip does far more of what you need. More answers per watt, per dollar. The price is rigidity. When the math shifts, a flexible GPU adapts.

The specialized chip may need a full redesign. You trade freedom for efficiency. So a chip just for AI throws away everything general. It keeps only multiply and add, packed tight.

Less flexible, but far better at the job that matters. Quick check now. One question is coming up. Let's see if it clicked.

How does a TPU's matrix engine work? Let's try to understand. In a simple way. The real work inside an AI model is multiplying big grids of numbers.

Those grids are called matrices. Rows times columns. To multiply matrices, you take a row and a column. Multiply them pair by pair, then add the products up.

That is one number of the answer. A full answer holds thousands of these numbers. Each one is a pile of multiply, then add. Do them in sequence, and the chip crawls.

So the TPU lays down a physical grid of tiny cells. One little multiply and add unit sitting at every spot. First it loads the weight matrix into the grid. One number is parked in each cell, and it stays put.

Then the input numbers march in from the left edge. Step by step, they flow across the grid like a wave. Every beat, each cell multiplies its parked weight by the number passing through. It adds that to a running total flowing down, then passes the input along.

This marching-data design has a name. A systolic array. The numbers pulse through the grid like blood through a beating heart. Here is the payoff.

Nothing is fetched from memory each step. Numbers get reused as they flow, so one clock tick fires the whole grid at once. So a TPU is a giant matrix calculator. Weights sit still, data flows through, and every cell multiplies and adds in step.

That is why it is so fast. Quick check now. One question is coming up. Let's see if it clicked.

How does chip making work? Let's try to understand. In a simple way. Every chip starts as a design.

Not a picture, but a map. It marks where each of billions of tiny switches and wires must sit. Here is the surprise. A chip is not carved from a block.

It is built up, one thin layer at a time, like printing a photograph. It all begins on a wafer. A thin, polished disc of ultra pure silicon. Every chip gets printed onto its surface.

First, each layer of the design becomes a mask. Think of it as a stencil holding the exact pattern for that layer. Coat the wafer in a light sensitive film. Shine light through the mask, and the pattern prints wherever the light lands.

This step is photolithography. Where the pattern printed, the wafer is etched away, or new material is laid down. That fixes one layer of wires into the silicon. Then repeat.

Print, etch, add. Do it dozens of times, stacking layer on layer, until the full circuit stands complete. And you never print just one. A single wafer is tiled with hundreds of identical chips, side by side.

Finally, the wafer is sliced into those separate chips. Each is sealed in a package with metal pins. Now it is a chip you can hold. So, from design to wafer.

Draw the pattern, make a mask, print it with light, then etch and repeat the layers. That is how sand becomes a chip. Quick check now. One question is coming up.

Let's see if it clicked.

How does the machine that prints chips work? Let's try to understand. In a simple way. The whole race is shrinking the switches.

Smaller switches mean faster, cheaper chips. But how tiny you can print comes down to one thing. Your light. Light works like a paintbrush.

You cannot draw a line thinner than its own wavelength. Ordinary light is too fat for today's features. So engineers reached for an extreme kind of light. Extreme ultraviolet, or EUV.

Its wavelength is about thirteen nanometers. Fine enough to draw the smallest lines. Now the twist. Only one company on Earth can build a machine that uses this light.

ASML, in the Netherlands. A true monopoly. How is this light made? Drops of molten tin are blasted by a laser fifty thousand times a second.

Each flash becomes plasma that glows in EUV. But EUV is swallowed by everything, even glass and air. So lenses fail. It must bounce off the flattest mirrors ever made, in a vacuum.

That beam reflects off a patterned mask, shrinks the image down, and stamps it onto the wafer. One layer of the chip, printed at a time. So here are the stakes. Without one of these machines, no cutting edge chip gets built.

Every advanced AI chip passes through ASML. So, to print smaller you need shorter light. EUV is made by zapping tin, steered by mirrors in a vacuum, and only ASML can build it. Quick check now.

One question is coming up. Let's see if it clicked.

How does Taiwan make the world's chips? Let's try to understand. In a simple way. The names on the chips, Nvidia, Apple, do not build them.

They only draw the design. Someone else does the making. That someone is a foundry. A factory that builds chips for other companies, but sells none of its own.

The biggest by far is TSMC, in Taiwan. So why does the making pile up in one place? A single cutting edge factory costs over twenty billion dollars and takes years to build. Few players can even try.

And there is a snowball effect. The factory that runs the newest process first learns its quirks first. That lead in know how only widens over time. It is also the neighborhood.

Thousands of specialist engineers and suppliers sit packed together around one science park. That web of skill is almost impossible to copy. One more reason. A foundry builds for everyone, but competes with no one.

So rivals can hand over their secret designs without fear. Trust keeps the customers coming. Add it up. Around ninety percent of the world's most advanced chips are made on this one island.

Nowhere else is even close. Which is the whole point. Every powerful AI chip, no matter who designed it, flows through Taiwan. That makes one small place a single point in the entire supply chain.

So, designers draw, a foundry builds. Huge cost, deep know how, and a tight cluster pushed the making into Taiwan, and the world's AI now leans on it. Quick check now. One question is coming up.

Let's see if it clicked.

How does one tiny flaw ruin a whole chip? Let's try to understand. In a simple way. A chip is not made alone.

Many identical chips are printed together on one round wafer of silicon. Later the wafer is sliced into separate squares. But the printing is never perfect. A speck of dust, or one flaw in a layer, can land somewhere on the wafer.

That spot is a defect. Here is the cruel part. A chip is one connected circuit. A single defect in the wrong place breaks it, and the whole chip is dead.

So not every chip survives. The share that come out working is called the yield. Good ones ship, dead ones are thrown away. Now the twist.

A bigger chip covers more of the wafer. So it is more likely to swallow a defect. Big chips get lower yield than small ones. And this is why huge chips cost so much more.

You do not just pay for the extra size. You pay for every neighbor a defect killed along the way. So designers fight back with spare parts. A chip is built with extra cores and memory rows.

If a defect kills one, a spare takes over. Sometimes a flaw only wounds a chip. So a partly broken one is sold cheaper, with the damaged cores switched off. This sorting is called binning.

So, tiny flaws are unavoidable, and one can kill a whole chip. Yield counts the good ones, big chips lose more, and spares and binning rescue the rest. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is an AI server? Let's try to understand. In a simple way. Picture the machine that trains a big model.

It is not a single chip. It is one box holding eight of the most powerful chips ever built. Each of those chips is enormous. It is pushed right to the size limit of what a factory can print, using the newest and most expensive process on Earth.

Around each chip sit tall stacks of special fast memory. This memory is scarce, hard to make, and only a couple of companies in the world can supply it. Now the chip and its memory stacks must be fused onto one slab. That advanced packaging step is a bottleneck.

Only a handful of lines on the planet can do it. The eight chips are then wired together with special high speed links, so the whole box behaves like one giant brain. Those links are custom, and they are not cheap. Add processors, networking, power, and heavy cooling.

Stack up every part, and one finished box lands near the price of a nice house. But the parts alone do not set the price. Everyone wants these boxes at once, while supply stays capped. Scarcity, not cost, drives the number up.

And you never buy just one. Training a frontier model needs thousands of these boxes, wired into one giant hall. The bill reaches into the billions. So an AI server is costly because it packs the largest chips, the rarest memory, cutting edge packaging, and worldwide scarcity into a single box.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is Moore's Law, and life beyond it? Let's try to understand. In a simple way. For fifty years, one pattern drove computing.

Every two years, engineers doubled the tiny switches on a chip. Same size, twice the power. We call it Moore's Law. But this was never a law of nature.

It was a trend, and trends end. Today those switches are only a few atoms wide. You cannot shrink much further. At that scale, physics fights back.

Electrons leak straight through walls too thin to stop them. That waste becomes heat, faster than we can carry away. So progress can no longer come from shrinking alone. One answer is to build upward.

Stack many chips into a tower, and stitch smaller pieces together like tiles. Another answer is to leave silicon behind. New materials, like carbon nanotubes, carry current with less waste and switch faster than silicon can. A bolder idea is to compute with light.

Beams of photons can carry signals and even do math, moving fast and running cool, with almost no leaking. And some drop the old design entirely. Quantum chips compute with fragile quantum states. Others copy the brain, wiring memory and math together, so data barely moves.

So the future is not one magic replacement. It is many small wins stacked together. Progress keeps coming, from clever design, not smaller switches. So, Moore's Law was the free ride of shrinking.

Now atoms set the floor, so we climb by stacking chips, new materials, and whole new kinds of computing. Quick check now. One question is coming up. Let's see if it clicked.