The economics of AI

What exactly is a training bill? Let's try to understand. In a simple way. Training a model is one big arithmetic job, paid for by the hour.

You rent thousands of chips, and the clock runs until the job finishes. So the real question is how much arithmetic. And the count is simple. Every parameter in the model gets touched by every token of training text.

So multiply the two together. Parameters times tokens. That product is the size of the bill. Now double the model, and keep the same text.

Every token touches twice as many parameters. Twice the arithmetic, twice the bill. But a bigger model on the same old text does not get much better. It mostly memorizes.

To earn its size, it needs more text, roughly in proportion. So both numbers grow at once. Ten times the parameters, and ten times the tokens. Ten times ten is one hundred times the arithmetic.

Now turn arithmetic into money. Chip hours, times the price per hour. A frontier run books tens of thousands of chips for weeks. And you pay all of it before a single user arrives.

A run that goes wrong still bills you for every hour it burned. So, the bill is parameters times tokens. Grow both together, and ten times bigger is not ten times the price. It is closer to one hundred.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the cost of one query? Let's try to understand. In a simple way. Normal software is almost free to serve one more user.

The server copies a page, and that copy costs a rounding error. AI does not work that way. When you send a question, nothing is looked up. The model computes the answer, and computing means running your words through every one of its billions of parameters.

And that happens once per word. The model writes an answer one word at a time, and each word is a fresh trip through the model. Two hundred words, two hundred trips. Every trip also reads everything so far.

Your question, anything you pasted, and the words already written. The longer the chat, the more work each new word takes. So a query is billed on two sides. Tokens in, and tokens out.

Output is the expensive side, because every output token buys its own pass through the model. And all that arithmetic runs on a chip you rent by the second. So the cost of one answer is simply the chip time it occupied. One answer might cost a fraction of a cent.

That sounds like nothing, until a million people ask something every day. That is the sting. Cost per user never drops to zero. Every extra user, even a free one, starts a meter that keeps running.

So, every query pays for its own arithmetic. One pass per word, over the whole model, on a chip rented by the second. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is the hidden cost of labeling data? Let's try to understand. In a simple way. Raw data is the cheap part.

Photos and text pile up almost for free. The money goes to the answer written next to each one. That answer is a label. A person looks at one example and decides.

Spam, or not spam. Tumor, or no tumor. A label cannot be copied. Every new example needs another human minute.

So the bill grows in a straight line with the dataset. Here is the first hidden part. Before anyone labels, someone writes the rulebook. What counts as spam, and where the edge sits.

Weeks of arguing, no labels yet. Then labelers disagree. The same item goes to three people, and an expert settles the split. You pay several times over for one trusted label.

The price depends on who must look. A crowd worker tagging photos costs cents. A radiologist or a lawyer costs tens of dollars for one item. Now change the rulebook.

Every label gathered under the old definition has to be redone. This is the rare cost you can pay twice for the same data. And it never ends. New slang, new products, new tricks.

So fresh labels keep getting bought. A subscription, not a purchase. And unlike chips, human attention does not get cheaper each year. That floor is why labeled data becomes the priciest line in the budget.

So, labels are hand made. One human judgment per example, plus the rulebook, the checking, and the redoing. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is an AI chip? Let's try to understand. In a simple way. An AI model does one tiny thing, over and over.

Multiply two numbers, add the result, repeat. Trillions of times for one answer. An ordinary computer chip has a few very clever cores, built to do different jobs one after another. Wrong shape for this work.

Here every sum is the same, and none of them waits for another. So you want thousands of plain cores, all working in the same instant. But the cores are the easy part. Each sum needs numbers pulled from memory, and the cores finish quicker than memory can deliver.

So they sit waiting. The fix is exotic memory, stacked in layers and bonded onto the chip itself. That stack alone is a large slice of the price. One chip is also not enough.

A big model is split across many chips, which swap numbers constantly, so each needs unusually fast wiring to its neighbours. Then there is making them. These chips need the newest, smallest manufacturing process, and only a couple of factories on earth can run it. They are also enormous by chip standards.

A bigger square of silicon catches more flaws, so more of every batch is scrapped. Hard to design, harder to build, wanted by everyone at once. So the answer to demand is a waiting list, not a discount. So, AI chips are expensive because the work demands thousands of cores, memory bonded on top, and a factory almost nobody can run.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the price of AI talent? Let's try to understand. In a simple way. Start with supply.

The people who have actually trained a frontier model from scratch number in the hundreds. Not the millions. And that pool stays small, because the skill comes from doing the run. You cannot pick it up from a paper.

You need the chips too. Now the demand side. A dozen well funded labs are chasing those same few hundred people. Fixed supply, many rich bidders.

That is an auction. And look at what one of these people decides. The data mix. When to stop.

Those calls steer a training run worth a hundred million dollars. So put the two numbers side by side. A few million in pay, guarding a hundred million in compute. Suddenly the pay looks cheap.

That is why the offer is mostly stock, and not cash. Stock that vests over four years. Cash buys a year. Stock buys the years.

And often a rival buys the whole team at once. The know how lives inside people's heads, not in the published papers. So it gets priced per head. Here is the trap.

Money orders more chips in a month. It cannot order more people who have already done this. So the price stays high. So, the price of talent is scarce supply, rich bidders, and enormous leverage.

A small salary line, sitting on top of a very large bill. Quick check now. One question is coming up. Let's see if it clicked.

How does selling API access work? Let's try to understand. In a simple way. You never hand over the model.

The weights stay on your machines. A customer gets an address, sends text to it, and gets an answer back. Every call through that address is measured. Tokens in, tokens out.

The price is public, so much per million tokens. Input and output carry different rates. Reading your question is cheap. Writing the answer costs more.

Now the business. Take what a thousand tokens sell for, subtract what they cost to serve, and the gap is your margin. Tiny per call, repeated constantly. Notice there are no seats.

Nobody buys a license. If a customer's app gets popular overnight, your revenue climbs with it, and nobody signs anything. And the door stays open. A card, a key, and the first call runs in minutes.

No sales meeting, so one price page serves a million developers. Then you post a ladder. A large model priced high for hard jobs, a small one priced low for easy ones. Buyers who would have left take the cheap rung.

But the money is lumpy. A few heavy customers make most of the bill, and nothing forces them to stay. And the price is one public number per token, so buyers compare providers on it. That number keeps falling, so volume must grow faster than price drops.

So, API revenue is a meter. Rent the model by the token, earn the gap on every one, and keep the meter spinning. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a monthly AI subscription? Let's try to understand. In a simple way. One price, every month.

Twenty dollars, say. The same number whether they open it once or a thousand times. But underneath, nothing is flat. Every answer you send burns tokens on your own machines.

Money comes in as a straight line, and goes out in bumps. So the plan rests on one number. The average cost per subscriber. Under twenty dollars, you make money.

Over it, every signup digs the hole deeper. And nobody is average. Most people ask a few questions a month. A small group asks all day, and one of them can cost more than the fee.

Which is what a flat fee really is. A pool. The quiet many pay for the loud few. So you defend the average.

A message cap. A smaller model once they pass it. A higher tier for the heaviest users. Why sell this way?

Because a meter makes people hesitate before typing. A flat fee removes that pause, so they use it daily. Daily use is what makes it stick. And your revenue becomes a line you can forecast.

Count subscribers, multiply by the fee. That number is what pays for the next model. The catch is the word monthly. Every month they decide again.

A few quiet weeks, and the card gets cancelled. So, a subscription is one flat price sitting on a bumpy cost. Watch the average, cap the extremes, and earn the habit. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is an AI feature worth? Let's try to understand. In a simple way. Start with something people already pay for.

A spreadsheet, an inbox. The AI part is not the product. It is a button inside one. That button rarely gets its own price tag.

Nobody buys the feature. What moves is the price of the whole product. So the payoff shows up as an uplift. The seat that sold for twenty five now sells for thirty.

Five dollars, every seat, every month. That is the trick. You already have the customers. Five dollars across a million existing seats, with nobody new to sell to.

Why pay more? Because the feature is judged against the hour it replaces. It drafts in seconds what took forty minutes. That hour carries a salary.

But nothing is metered on their side. Every draft burns tokens on yours. So the uplift per seat has to beat what serving it costs. Which makes adoption the dial.

If one in twenty clicks it, your token bill stays small, and so does the reason to pay more. The second payoff is quieter. A feature used daily makes leaving harder. Fewer cancellations is money too, it just never shows as a new line.

And the uplift has a clock on it. Once every rival ships the same button, it stops adding money and only keeps what you have. So, an AI feature is not a product. It is a lift on a price you already charge, across a base you already own.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the chipmaker's cut of the AI boom? Let's try to understand. In a simple way. Start with what they sell.

Not an AI product. The machines every lab needs to build one. The tool, not the product. They get paid at purchase.

A lab buys chips years before its model earns anything. If the model flops, the chips are already sold. Nearly every buyer wants the same kind of chip, and only a couple of designs are good enough. All that demand lands on two or three sellers.

The price is not set by the parts. It is set by the queue. A chip costing a few thousand dollars to build sells for tens of thousands. Designing that chip costs billions, once.

After that, every copy is just manufacturing. So each extra chip sold is almost pure profit. They also stopped selling single chips. You buy the whole rack.

Chips, memory, wiring, switches, software. One sale, a much bigger invoice. Years of AI code is written for one seller's toolkit. Switching chips means rewriting it, so the next order comes back to the same door.

And when supply is short, the seller picks the buyer. Customers pay in advance just to hold a place in the line. Notice what this revenue is. Orders.

A buyer can pause a year of spending in one meeting, and the line drops with it. So, chipmakers reach the boom's money first. Scarce tools, sold up front, priced by the queue, and hard to walk away from. Quick check now.

One question is coming up. Let's see if it clicked.

How does renting AI power from a cloud work? Let's try to understand. In a simple way. Who owns the machines?

Not the lab. A cloud buys the chips, racks them in a building, and pays the power bill. Then it rents them out by the hour. Ask for eight chips, get them in minutes, and the meter runs until you give them back.

Where does that price come from? Take what the machine cost, divide by the hours it works before it is old, add power and margin. So the business rests on one number. How many of those hours actually sell.

An idle chip earns nothing and still costs everything. For the renter, the trade is plain. Rent costs more per hour than owning. But you pay for three weeks, not three years.

AI demand is spiky. A training run wants a thousand chips for a month, then none. Buy for that peak and it sits idle all year. The same hour sells three ways.

Reserved for a year, cheapest. Wanted right now, dearer. Leftover time, cheap, and taken back when someone pays more. You are not renting only chips.

You rent the wiring between them, the cooling, and a connection to the grid. Electricity is the real bottleneck. Clouds also rent the model itself. Someone else's weights, on their machines, billed per token.

The landlord earns without training anything. So, a cloud is a landlord for computing power. It owns the metal, sells the hours, and lives on how few sit empty. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the economic tradeoff of open source AI? Let's try to understand. In a simple way. Open here means the weights are published.

The finished file from a run that cost hundreds of millions, downloadable by anyone, for nothing. So the copy price is zero. Nobody pays a licence fee, and a copy costs nothing to make. That revenue line does not exist.

Why give it away? Because the giver sells something next to the model. Chips, cloud, a product. A free model makes those more valuable.

What they buy is adoption. Developers learn their model and build tools on it. Downloads instead of dollars, and their design becomes the default. Now the other side.

Free weights are not a free service. You download the file, then you rent the machines and hire the engineers. The bill moved, it did not vanish. Instead of a fee per call, you carry a fixed monthly cost, busy or idle.

Which is why volume decides. Light and bursty, renting is cheaper. Heavy and steady, that fixed cost spreads thin and self hosting wins. You also buy control.

The price cannot be raised on you, the model cannot be retired, and your data never leaves your own machines. The cost of that control is quality. Open weights usually trail the best closed model by about a year. You trade capability for independence.

So the tradeoff is simple. The giver swaps licence money for reach. The user swaps a monthly bill and some quality for control. Quick check now.

One question is coming up. Let's see if it clicked.

How does getting cheaper at scale work? Let's try to understand. In a simple way. A service pays two kinds of bill.

The arithmetic for each answer, and a pile paid once, however many people show up. That once paid pile is divided by every query served. A hundred million dollars over ten thousand queries is brutal. Over ten billion, it vanishes.

Now the part people miss. To answer anything, a chip reads the whole model out of memory. That read costs the same for one question or fifty. So the server waits a few milliseconds, gathers the questions that just arrived, and runs them together.

One read of the model, fifty answers out. With few users that batch is nearly empty. Same read, one answer back. Traffic is what fills the seats.

You also buy chips for the busiest minute. With a thousand users that peak towers over the average. With ten million, the spikes cancel. More users bring more repeats.

The same popular question, the same instructions before every request. Store that work once, and a hit skips the model. And shaving one percent off each answer is worthless at a thousand queries a day. At a billion, it pays a whole team.

But the curve bends, it never reaches zero. Every batch still burns chip seconds. Scale spreads the fixed part. It cannot delete the arithmetic.

So, scale divides the fixed bill, packs every batch, and smooths the traffic. Cost per query falls fast, then flattens. Quick check now. One question is coming up.

Let's see if it clicked.

How does shrinking a model work? Let's try to understand. In a simple way. Every answer runs through every number in the model.

So the cost of one answer is set by how many numbers there are. So turn that dial down. Not the traffic, the model itself. One tenth the size, about one tenth the arithmetic.

The main way is distillation. The big model answers millions of questions, and a small model is trained to copy those answers. Copying is easier than discovering. The student is shown the finished answer every time, so a much smaller model picks up most of the same behaviour.

There is a second trick. Store each number in fewer bits. Four instead of sixteen makes the same model four times lighter. Now the money.

A model that fits on a single chip stops needing eight of them wired together. The extra chips go. The chip you keep still costs the same by the hour. A lighter model gets more answers out of that same hour.

And the shrinking is paid once. One distillation run, then every call after it is cheaper. That is why labs ship a small model too. The catch is quality.

On everyday requests the small model keeps up. On long reasoning and rare facts, it slips. So services keep both, and send the easy majority to the small one. Shrink the model, pay once, and every answer after it costs less.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a data moat? Let's try to understand. In a simple way. Start with what a rival can simply buy.

The design is printed in papers. The chips are on sale. The engineers can be hired away. One thing has no price tag.

The record your product writes every day. What your users asked, and what they did with the answer. Text scraped from the open web is a commodity. Your competitor trained on the very same pages, so it hands nobody an advantage.

Your log is different because it is a side effect of use. Every edit a user makes is a correction, arriving for free while you run. And it compounds. A better model wins more users, more users write more log, and the next model trains on that.

The loop feeds itself. So the gap is priced in time, not money. A rival with equal funding still cannot buy five years of somebody else's usage. And you are still collecting while they chase.

Starting behind, they have to win users faster than you just to stop the distance growing. The pile need not be huge. Ten thousand rows from your own factory floor beat a billion generic ones, because they match the actual job. But a moat drains if you stop.

Staff leave, data gets sold, habits change. A log nobody refreshes slowly stops describing today. So the moat is not the model. It is the record only your own users can write, and only time can copy.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a winner take most market? Let's try to understand. In a simple way. Start with the price of showing up.

The training run, the data, the machines. All of it, paid before your first customer arrives. Now, how buyers choose. Everyone reads the same short list of scores, and nearly all the traffic goes to whatever sits on top.

And leaving is easy. Changing provider is one line in a config file. So when the lead moves, the traffic moves with it. So close is not enough.

A model two percent behind does not earn two percent less. It earns what a second choice earns. Then the money loops. The winner's income funds the next run, bigger than the last.

The one behind must fund the same run from less. The scarce things go to the leader too. A queue for the newest chips. A few hundred people who have run a job this big.

Meanwhile the world builds around the leader. Tools and tutorials assume its interface. That is not loyalty, it is furniture, and furniture is heavy. So why not exactly one?

Buyers refuse. A single supplier means no lever on price, so big customers keep a rival alive on purpose. And the lead is rented, not owned. It rests on one training run, and the next one belongs to whoever guesses the better design.

So the shape is a handful, not a crowd. A huge ticket to enter, one ranking, and traffic that piles onto today's leader. Quick check now. One question is coming up.

Let's see if it clicked.