Chain-of-Thought, dissected

What exactly is a forward pass? Let's try to understand. In a simple way. A large language model is an AI that writes text one small piece at a time.

Each piece is called a token: a word, or a chunk of one. So how does it pick the next token? Inside the model sits a stack of layers. Your text enters at the bottom and passes through every layer once, in order.

That one trip, bottom to top, is called a forward pass. At the top, out comes a score for every token the model knows: how likely each is to come next. The model picks one token from those scores. And then, here is the part people miss.

It stops. Completely. One token out, and the machinery goes quiet. Now the key fact.

The number of layers is fixed when the model is built. Say ninety six layers. Then every token costs exactly ninety six stages of processing. Never more, never less.

Ask what colour the sky is, or ask a problem needing eight careful steps. Same ninety six layers, either way. There is no inner loop that keeps running until an answer is found. One thing does vary.

With a longer prompt, each layer looks across more tokens, so a pass costs more work. But the depth, the count of layers, never grows. The precise phrase is fixed depth. So hold this picture.

One token, one pass through a fixed stack. This entire course leans on that single fact. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is autoregression? Let's try to understand. In a simple way. Last time, we saw one forward pass produce one token.

But answers are long. So how does a model write a whole paragraph? Here is the trick. The model writes a token, then glues it onto the end of the input, and runs a fresh forward pass on the longer text.

This loop has a name. Autoregression. The model's own output keeps becoming its next input, token after token, until the answer is done. Say the prompt is, the capital of France is.

The model emits, Paris. Now the input reads, the capital of France is Paris. The next pass continues from there. Now the strange part.

On the way back in, there is no marker saying, you wrote this. Prompt text and generated text look identical. Just one stream of tokens. That stream is the context window.

The running text the model can look back over. Every new pass reads all of it, your words and its own words alike. And generation is strictly left to right. The model does not plan the whole answer and then type it out.

Each token is chosen blind to every token that will follow it. And remember, one pass is a fixed amount of computation. But a fifty token answer gets fifty passes. Hold that thought.

It becomes the heart of this paper. So, autoregression. Write a token, feed it back, run again. Everything the model writes joins the input for everything that comes after.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is in-context learning? Let's try to understand. In a simple way. Everything a model knows is stored in its weights.

Billions of numbers inside the network. Every forward pass runs your tokens through those same numbers. Training is how those numbers get set. Reading mountains of text nudges the weights, millions of times.

Then training ends. The weights freeze. Now the surprise. Take that frozen model.

Put a few examples of a brand new task into the prompt. It starts doing the task. That is in-context learning. Try it.

The prompt says, apple becomes pomme. House becomes maison. Cat becomes. The model writes chat.

Those examples arrived seconds ago, as plain tokens in the context. How? Remember autoregression. Every new token is predicted from everything already sitting in the context, examples included.

So the pattern in the prompt steers each prediction. And through all of it, not one weight changes. Clear the prompt, and the skill is gone instantly. The adaptation lives in the context, never in the network.

This trick was made famous in twenty twenty by the GPT-3 paper, titled Language Models are Few-Shot Learners. It is the foundation of our paper. The authors state it plainly. No model got extra training in this work.

Every result comes from an off-the-shelf model, adapting from its prompt alone. So hold this picture. Weights are long-term knowledge, locked at training time. The prompt is where the model adapts, live.

Chain of thought, writing reasoning steps before the answer, happens in that second place. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is few-shot prompting? Let's try to understand. In a simple way. Last lesson: in-context learning, the model picks up a task from its prompt, no training.

Few-shot prompting is the standard recipe. You put a handful of worked examples before your real question. Each example is called an exemplar. Here, an exemplar is a question paired with its answer.

Question, answer, question, answer, then yours. The model continues the pattern and answers directly. So what is the model taking from those examples? The obvious guess: it studies them like flashcards, learning the right answers.

A twenty twenty two study by Min and colleagues tested that guess directly. They took the exemplars and swapped the correct answers for random ones. Here is the surprise. Performance barely dropped.

Across twelve different models, GPT-3 included, on classification and multiple-choice tasks, wrong answers in the examples hardly hurt. So what does matter? Three things. The set of answers the task allows, its labels.

The kind of text in the inputs. And the overall format, the shape of the sequence. In other words, exemplars are a demonstration, not a fact sheet. They show what a valid turn looks like.

Picture sentiment tagging: is a sentence positive or negative? Flip half the labels in your exemplars to wrong ones. Accuracy barely moves. The format taught the job, not the labels.

Hold onto this. The paper we are dissecting changes exactly one thing: what the exemplars demonstrate. If exemplars teach format, you can teach a richer format. That comes next.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly are the math problems that stumped giant models? Let's try to understand. In a simple way. In twenty twenty one, researchers published a benchmark called GSM8K.

A benchmark is just a fixed exam of problems, so every model is measured the same way. It holds eight and a half thousand grade school word problems, written by people. One thousand are held out for testing. Here is one in that spirit.

A bakery bakes forty muffins, sells twelve, bakes twenty more, then sells half of what is left. How many remain? You solved it in steps. Forty minus twelve is twenty eight.

Plus twenty is forty eight. Half of that, twenty four. Three small moves, chained. That is the design.

Every problem needs two to eight steps, and each step is elementary. Add, subtract, multiply, divide. So the hard part is not the arithmetic. It is reading the story and picking the right operations in the right order.

The plan, not the sums. Now the strange part. Bigger models had been fixing task after task, but not this one. PaLM, the biggest model in this paper, shown a few question and answer examples, solved fewer than eighteen problems in every hundred.

The best score anyone had was fifty five percent. And that took extra training on thousands of worked solutions, plus a second model to check answers. So hold this picture. Easy pieces, giant models, stubborn failure.

The pieces are not the problem. The chaining is. That is where we go next. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the single-pass bottleneck? Let's try to understand. In a simple way. Recall lesson one.

To produce one token, the model runs the input through its stack of layers once, then stops. That trip is the forward pass. Here's the catch. The layer count was fixed when the model was built.

Say ninety six. An easy question gets ninety six layers. A brutal one gets the same. And there's no internal loop.

The model can't say, let me go around again. One trip. That's the budget. Now try one.

Maya buys four boxes of six pens, gives four away, then splits the rest between two friends. How many does each get? Watch the structure. Four times six, twenty four.

Minus four, twenty. Split two ways, ten each. Each step needs the previous result. You can't split the rest before you know it.

Steps like that are sequential, and sequential work eats layers. Researchers call this serial depth. More chained steps than depth, and one pass runs out of road. A longer prompt gives the pass more to read.

But that's width, not depth. Looking wider isn't stepping deeper. Remember lesson five. These word problems take two to eight chained steps.

Easy arithmetic. The chaining is the trap. One honest note. This is intuition, not something the paper proved.

A later lesson puts it to the test. So here's the tension. Answering immediately gives the model one fixed pass for the whole chain. But each token written buys a fresh pass.

Hold that thought. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a chain-of-thought exemplar? Let's try to understand. In a simple way. You know few-shot prompting by now.

You show the model a few worked examples, each one a question and its answer, and it copies the pattern. The paper changes exactly one thing. Between the question and the answer, it inserts the reasoning, written out in plain sentences. So each exemplar becomes a triple: question, steps, answer.

That middle part is the chain of thought. A series of short natural language steps that walk from the question to the final answer. Let's build one. Maya has four pencils.

She buys three packs, with five pencils in each pack. How many pencils does she have now? Line one just grabs a fact from the question. Maya starts with four pencils.

No math yet. It simply puts one useful number on the page. Line two does one small operation. Three packs of five is fifteen pencils.

One multiplication, nothing more, and the result is now written down. Line three combines what the earlier lines produced. Four plus fifteen is nineteen. And only then comes the closing line.

The answer is nineteen. Answer last, always. Notice the anatomy. Every line does exactly one thing, in order, and each result feeds the next line.

Small steps, chained, until the answer falls out. In the paper, the authors hand wrote eight of these exemplars. No careful tweaking of the wording, no search for magic phrasing. Eight worked examples in the prompt.

That is the entire method. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is scratch memory? Let's try to understand. In a simple way. Quick recall.

A model runs its layers once per token, top to bottom. The depth never grows, no matter how hard the question is. And remember autoregression. Every token the model writes is appended to the context, and read again on the very next step.

Put those together and you get the intuition. Written steps act like memory. A partial result, once written, sits on the page. Nothing has to hold it.

Take a small problem. A bakery bakes three trays of twelve rolls, then sells twenty. The model writes, three times twelve is thirty six. The next pass just reads thirty six and subtracts twenty.

People call this a scratchpad. But notice, it's not private. The steps are ordinary output text, sitting in the context window, where every later pass can read them. And it buys computation.

Answer immediately, one pass. Write steps first, and you get one full pass per token written. Harder problems can, in principle, get more work. Now, here is the thing.

This is an intuition. The paper states it informally, as a reason chain of thought might help. It proves nothing by itself. Because rival stories fit.

Maybe any extra tokens would help, even meaningless ones. Maybe reasoning text just nudges the model toward the right knowledge, and the steps themselves do nothing. So the claim needs testing. Change one thing at a time, keep the rest fixed, and see what survives.

Those experiments are coming soon. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is emergence? Let's try to understand. In a simple way. Here is the puzzle.

Chain of thought is just text in a prompt. It should help any model, right? It doesn't. Quick term first.

A model's size is its parameter count, the learned numbers inside it. They tested from a few hundred million up to five hundred and forty billion. The result is brutal. Below roughly ten billion parameters, chain of thought usually made models worse than asking directly.

On grade school math, LaMDA at eight billion parameters fell from three point two percent to one point six. PaLM, at five hundred forty billion, jumped from seventeen point nine to fifty six point nine. The gains only showed up in models around one hundred billion parameters. Same prompt, same eight exemplars.

Only size changed. That pattern has a name. Emergence. Absent as models grow, then suddenly present past a size threshold.

Not a smooth slope. A jump. So what were small models writing? The authors looked.

Fluent, grammatical chains of thought that were simply illogical. The format is perfect. The content is nonsense. And remember autoregression.

Every token written gets fed back in. A wrong step becomes input, and the model builds on it. Exemplars taught the format, not the logic. Fluency comes cheap.

Logic does not. Some small models drifted, or repeated themselves, and never reached a final answer at all. So, the takeaway. Scratch memory only helps if the steps you write are right.

Below the threshold, they mostly aren't. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is an ablation study? Let's try to understand. In a simple way. Last time we said written steps act like scratch memory.

Nice story. But a nice story is not evidence. The authors put it on trial. An ablation study is simple.

Take your method, delete one ingredient, and measure again. If the score collapses, that ingredient was doing the work. Rival explanation one. Maybe the words are decoration, and only the math matters.

So they prompted the model to write just the equation. No sentences at all. On grade school word problems, the largest LaMDA scored five point four percent that way. Plain prompting, the baseline, scores six point five.

The full chain? Fourteen point three. The words carry weight. Rival two.

Maybe extra tokens just buy extra compute, and content is irrelevant. So they had the model print dots. Same number of tokens, zero meaning. Result: six point four percent.

Dead flat with the baseline. So here is the correction to our scratch memory story. Writing tokens only helps when the tokens mean something. Rival three.

Maybe the prompt style just wakes up knowledge the model already has. Test: put the reasoning after the final answer in every exemplar. Six point one percent. Flat again.

Remember, generation runs left to right. A chain written after the answer cannot feed into it. The chain must come first. So three boring explanations, three flat lines.

What survives is the interesting one: meaningful steps, written before the answer, each one feeding the steps that follow. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a faithful chain of thought? Let's try to understand. In a simple way. By now you know the chain works.

It reads like the model showing its work. Tempting to treat it as an explanation. Here is the key word: faithfulness. A chain is faithful if it truly describes how the model reached its answer.

Helpful and faithful are different claims. The authors knew this. They said the chain suggests how the model might have answered, and left whether it's actually reasoning an open question. The paper itself found a crack.

Of fifty correct answers checked, one chain reached the right answer through wrong reasoning. The two can come apart. In twenty twenty three, Turpin and colleagues rigged prompt examples so the answer was always option A. Models followed that hidden pattern.

Their chains never mentioned it. Instead, the chains gave confident reasons for the biased answer. Plausible, but misleading. When pushed toward wrong answers, accuracy dropped by as much as thirty six percent.

Lanham and colleagues tried edits: cut the chain short, slip in a mistake. If the answer doesn't change, the chain wasn't doing the work. And larger models often depended on their chains less. So the danger is trust.

A tidy chain makes the answer feel verified. But it's just text, generated token by token like everything else. The honest position: chain of thought is an accuracy method. Proven.

It is not a guaranteed window into what the model computed. Sometimes it explains. Sometimes it decorates. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the reasoning era? Let's try to understand. In a simple way. Wei and colleagues left doors open.

Their method sampled one chain per question, always taking the most likely next token. And it needed eight worked examples. Two months later, the same group added self-consistency: sample many chains per question, then take the majority final answer. On grade school math, nearly eighteen extra points.

The intuition: hard problems have many routes to the right answer, but wrong chains scatter. Agreement is a signal. Then Kojima and colleagues dropped the exemplars entirely. Append one instruction, let's think step by step, and a big model produces the reasoning format on its own.

Remember lesson four? Exemplars teach format, not fact. This is the proof. In a large enough model, a short instruction can request that format directly, no examples needed.

Yao and colleagues asked, why one chain at all? Tree of Thoughts proposes several next steps, scores them, and can backtrack. On one hard puzzle, chain of thought solved four percent. The tree reached seventy four.

Then the biggest shift. Models were trained to reason: generate a chain, check the final answer, reward the model when it lands. That's reinforcement learning. o one led the way.

DeepSeek R one made it public in twenty twenty five: pure reinforcement learning, no human written chains needed, and behaviours like self checking emerged during training. So the arc: eight worked examples in a prompt became a training objective. The prompt trick became the product. That is the reasoning era.

Quick check now. One question is coming up. Let's see if it clicked.