Reasoning models

What exactly is a reasoning model? Let's try to understand. In a simple way. Here is how a normal LLM answers.

You ask, and it starts writing immediately. First word, right away. No pause, no scratch work. One single pass.

That works for easy questions. But try a tricky logic puzzle, or a multi step math problem. Guessing the answer in one shot? That usually goes wrong.

Think about yourself. What is seventeen times twenty four? You don't just blurt it out. You grab scratch paper, work in steps, and check as you go.

A reasoning model does exactly that. Before giving you the answer, it works the problem through, step by step. Deliberate first, answer second. That's the whole definition.

So the output has two phases. First a working phase: the model writes out a chain of thought, its intermediate steps. Then a short final answer built on that work. And here is the powerful part.

In that working phase, the model can catch its own mistakes. Wait, that's wrong. Backtrack. Try a different path.

Is it a new kind of brain? No. Underneath, it's the same transformer, predicting one token at a time. What changed is the behavior: it deliberates before it commits.

You've probably met these models. OpenAI's o series, DeepSeek R1, Claude in extended thinking mode. They all pause, work it through, then speak. So, the gist.

A standard LLM answers in one pass. A reasoning model earns its answer: it works step by step, checks itself, and only then replies. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly are thinking tokens? Let's try to understand. In a simple way. Here is the thing.

A language model makes one token at a time, and every token gets the same fixed slice of computation. So an instant answer gets exactly one shot. Now try this. Multiply thirty seven by twenty four in your head.

Hard, right? On paper, easy. Reasoning models pull the same trick. They write out steps first.

Those written steps are thinking tokens. Ordinary tokens, generated the usual way. But they are scratch work, not the answer. The literature calls this chain of thought.

First this, so that, therefore this. Each step lands on the page, and the next one builds on it. Why does writing help? Every token the model writes gets fed back in as input.

Intermediate results are stored in the text itself. The text becomes working memory. And a second effect. Each new token triggers a full pass through the network.

Ten steps of thinking means ten more chances to compute. In products you often never see this. The thinking streams into a hidden block and gets stripped from the reply. That pause when a model says thinking?

It's writing tokens you never read. One catch. The model trusts its own thinking like any other text. A wrong step gets built on, not questioned.

So it helps, but it is not a truth guarantee. So, thinking tokens are ordinary tokens used as scratch work. Write the steps, feed them back, then answer. Reasoning out loud, literally.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is test-time compute? Let's try to understand. In a simple way. Here is the big picture.

You can spend compute on a model in two places. Once, during training. Or every time it answers. That second one is test-time compute.

For years, all the money went into training. Bigger model, more data. At answer time, the model made one quick pass and stopped. Fast, but fixed.

Now here is the shift. Keep the model exactly the same. Just let it generate more tokens of reasoning before it commits to an answer. More compute, at test time.

Take a hard olympiad problem. Forced to answer in ten tokens, the model guesses, and it's wrong. Given ten thousand tokens to explore and check, it often nails it. And this is a dial you control.

APIs now expose it as a reasoning effort setting. Low, medium, high. Same weights, different thinking budget. So why call it buying?

Every reasoning token is billed like any output token. Better answers cost real money, and real waiting time. Here is the striking part. Accuracy climbs smoothly as the thinking budget grows.

Ten times more compute, another step up. It behaves like a scaling law, but at inference. That opens a trade. On many problems, a small model thinking for long can beat a much bigger model answering instantly.

Compute at answer time substitutes for size. So, test-time compute means spending extra computation while answering, not just while training. You pay per token of thought, and you get accuracy back. Quick check now.

One question is coming up. Let's see if it clicked.

How does reasoning training work? Let's try to understand. In a simple way. A base model just predicts the next word.

It doesn't naturally stop, think, and check itself. That behavior has to be trained in. So how do you teach it? Here is the problem.

You can't hire people to write millions of perfect step-by-step solutions. Too slow, too expensive. So labs use a different trick. They pick problems where the answer can be checked automatically.

Math has one right number. Code either passes the tests or it doesn't. No human grader needed. Now the training loop.

Give the model a hard problem. Let it generate a long chain of thought, then an answer. Do this many times per problem. Then check each answer.

Correct ones get a reward. Wrong ones don't. And reinforcement learning nudges the model toward whatever thinking led to the right answers. This recipe has a name.

Reinforcement learning with verifiable rewards. Notice what's graded: only the final answer. The steps in between are never labeled by anyone. And here is the magic.

Useful habits emerge on their own. The model starts double checking, backtracking, saying wait, let me reconsider. Nobody wrote those moves in. In practice there's one more step.

Before the RL, models often get a short supervised warm-up on a few good reasoning examples. It keeps the thinking readable. So, reasoning is trained by trial and reward. Sample lots of attempts, keep what gets checked correct, and long careful thinking grows out of that.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the right time to use a reasoning model? Let's try to understand. In a simple way. Here is the thing.

A reasoning model bills every thinking token like an output token. A three line answer can hide thousands of thinking tokens. You pay for all of it. So when does that spend pay off?

On multi step problems. Math, debugging code, planning, hard logic. Anywhere the answer depends on a chain of steps, and one slip ruins everything. Compare two questions.

Schedule four meetings across three time zones with no clashes. That's stepwise work, so thinking helps a lot. Now ask for the capital of France. No steps.

Pure recall. For recall, a reasoning model still burns thinking tokens. Same for summaries, classification, casual chat. Accuracy barely moves, but the bill can grow ten times or more.

Researchers call this overthinking. On easy questions, reasoning models generate long chains anyway. Sometimes they even second guess a correct first answer and talk themselves into a wrong one. Money is not the only cost.

Thinking tokens come out before the answer starts, so the user just waits. Thirty seconds of silence is a real product problem. So in practice, teams route. A cheap fast model handles easy traffic, and only hard multi step queries escalate to the expensive reasoner.

This is called model routing, or a cascade. Here is the rule. If the answer needs a chain of checkable steps, pay for thinking. If it's recall or routine text, don't.

Match compute to the question. Quick check now. One question is coming up. Let's see if it clicked.

How does reasoning distillation work? Let's try to understand. In a simple way. Reasoning models are great, but the best ones are huge.

Huge means slow and expensive. So here is the question. Can a small model learn to reason too? Distillation is the answer.

You take a big model, the teacher. And a small model, the student. The student learns by copying the teacher's work. And here is the key part.

The student does not just copy final answers. It copies the whole chain of thought. Every step of the reasoning, written out. So the recipe is simple.

Ask the teacher thousands of hard questions. Let it think out loud. Save the full traces. That becomes your training data.

One catch. Teachers make mistakes too. So you check each trace and keep only the ones that reach the correct answer. This is called rejection sampling.

Then you train the student on those traces with plain supervised fine-tuning. No reinforcement learning needed. The student simply learns to predict the next token of good reasoning. Does it work?

Yes, surprisingly well. DeepSeek distilled its big reasoning model into students thirty times smaller. Those small models beat much larger ones on math. But there is a ceiling.

A student mostly imitates. It rarely beats its teacher. And if it copies a flawed habit, it learns the flaw too. So, distillation means a big model thinks out loud, and a small model learns to imitate those traces.

Big reasoning, small price. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is self-consistency? Let's try to understand. In a simple way. Ask a model a hard math question once, and it writes one chain of thought.

If that chain slips anywhere, the final answer is wrong. One try is fragile. Self-consistency fixes this. You ask the same question many times, with sampling turned on, so each run takes a different reasoning path.

Then here is the trick. You ignore the reasoning and look only at the final answers. Whichever answer shows up most often wins. A simple majority vote.

Say you sample ten chains on a word problem. Six say twenty four, three say eighteen, one says forty. You output twenty four, and you're usually right. Why does this work?

Wrong chains fail in scattered ways, so their answers spread out. Correct chains keep landing on the same answer. Voting surfaces that agreement. A cousin trick is best of N.

Sample many answers, then let a separate verifier, or reward model, score each one. Keep the highest scorer instead of voting. One catch. Voting needs answers you can match exactly, like numbers.

For open ended writing, there is no majority to count, so you need that verifier instead. And it isn't free. Ten samples cost ten times the compute. The gains are real but they flatten out fast, so most people stop around five to ten samples.

So, self-consistency means sample many reasoning paths, then trust the answer they agree on. Agreement is a cheap stand in for correctness. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is the limit of machine reasoning? Let's try to understand. In a simple way. Reasoning models can feel almost human.

Careful chains, step by step. But underneath, it is still next token prediction. And that comes with hard limits. Take a math problem the model solves perfectly.

Now swap the names, or change the numbers. Accuracy drops. A true reasoner would not care. A pattern matcher does.

It gets worse. Add one irrelevant line, like five kiwis were smaller than average. Models start subtracting those kiwis. The GSM Symbolic study showed exactly this.

Next, faithfulness. The written chain of thought is not always the real computation. Sometimes the model decides first, then writes a tidy story to justify it. That is an unfaithful explanation.

Chains are fragile too. One wrong step early, and everything after builds on a false fact. Longer chains mean more chances to slip. Now the strangest one.

Push puzzle difficulty up, and accuracy does not fade gently. It collapses past a threshold. Even with plenty of thinking budget left unused. Worse, the model cannot reliably tell when it failed.

Wrong answers arrive in the same confident tone. There is no built in truth detector. So what is going on? These models generalize from training patterns.

Near those patterns, they shine. Far from them, on truly novel structure, they break. So, machine reasoning is powerful but brittle. Sensitive to wording, sometimes unfaithful, and it hits a wall on hard novel problems.

Trust it, but verify it. Quick check now. One question is coming up. Let's see if it clicked.