← back to the archiveCover illustration for “Reasoning model: an LLM that thinks before it answers”
TERM7mo ago·Published · Updated ·by Andy Padia

Reasoning model: an LLM that thinks before it answers

In short: A reasoning model is an LLM that spends hidden reasoning tokens working through a problem before it answers, trading extra cost and delay for accuracy.

A reasoning model is a language model that generates internal reasoning tokens before it produces its answer, spending extra computation to plan, check and revise on hard problems. OpenAI's reasoning guide, read in November 2026, says these tokens are hidden from the API output, still take up context window space and are billed as output tokens. OpenAI and Anthropic both expose a setting that turns the thinking up or down.

Reasoning model in plain words

Think of a colleague who says "give me a minute" and works it out on paper, compared with one who answers off the top of their head. The first is slower and pricier, and better on the hard questions.

How it works

The model produces reasoning tokens first, often hidden or summarised for the user, then the final answer. You control the depth with an effort level or a token budget. OpenAI's guide lists effort values from none up to max, depending on the model.

Anthropic's extended thinking documentation describes the older fixed-budget style, with a minimum budget of 1,024 tokens, and now marks it as legacy in favour of adaptive thinking steered by an effort setting. The same page states the trade-off plainly: higher budgets allow more thorough reasoning, with diminishing returns and increased latency.

Example

As an illustration, take a race condition. A standard model suggests adding a lock in the obvious place. A reasoning model traces the order of events across two handlers, spots that the lock is released before the callback runs, and proposes a different fix with a test that reproduces the timing.

Where you'll meet it

  • Model families that offer thinking or reasoning-effort settings
  • Coding agents, which often use reasoning for planning steps
  • Maths, science and analysis benchmarks
  • Model routing setups that send only hard queries to reasoning mode

What it changes for you

For developers, the question becomes "how much thinking does this task deserve?". Routing simple requests to fast settings and hard ones to deep reasoning keeps cost under control, because the hidden tokens are billed. For product teams, latency expectations change: users wait longer for some answers, so show progress.

Reasoning model vs chain-of-thought prompting

Chain-of-thought asks any model to show its steps in the visible answer. A reasoning model does the stepwise work in its own reasoning tokens before the answer starts.

Common mistake

My view: switching everything to maximum reasoning because benchmark charts look better. On extraction, formatting and simple lookups, it adds cost and delay with no gain. Measure per task type.

Chain-of-thought · Model routing · Cost per task · Time to first token · Large language model

From the archive: Put reasoning at the workflow's uncertain decisions

Measure how much reasoning each task type needs before you pay for the maximum everywhere.

#field-guide#models#reasoning#llm

Sources

  1. developers.openai.com/api/docs/guides/reasoning
  2. platform.claude.com/docs/en/build-with-claude/extended-thinking
← older drop
Agent skills: packaged know-how an agent loads on demand
newer drop →
AI gateway: one control point for every model call

related drops

explore all 557 drops →
← back to the archiveday 175