← back to the archiveCover illustration for “Context window: how much an LLM can read at once”
TERMday 161·2w ago·Published · Updated ·by Andy Padia

Context window: how much an LLM can read at once

In short: The context window is the most text, in tokens, a model can consider in one request: what fits, what it costs and why bigger isn't always better.

The context window is how much input and output, measured in tokens, an LLM can read and write in a single request. Everything the model uses to answer, including system instructions, conversation history, documents, tool results and its own reply, must fit inside it. As of November 2026, Anthropic's documentation lists windows of up to 1M tokens on current Claude models.

Context window in plain words

The size of the desk. You can only work with the papers that fit on it; the rest are in the filing cabinet, invisible until someone brings them over.

How it works

Text is split into tokens, roughly three-quarters of an English word each. The model attends to all tokens in the window when producing each new token. Larger windows let you include more material, but cost and latency usually rise with input length, and attention to any single detail can weaken as the window fills. Anything outside the window simply does not exist for that call.

Example

As an illustration, a coding agent session: system prompt and tool definitions, the CLAUDE.md file, five source files, test output and twenty turns of history all share one window. When it nears the limit, the harness summarises or drops older turns, often called compaction.

Where you'll meet it

  • Model specification pages listing window sizes
  • "Context full" or compaction messages in coding agents
  • Pricing, which charges per input and output token
  • RAG design decisions about how many chunks to include

What it changes for you

For developers, the window is a budget to spend deliberately. For architects, large windows reduce but do not remove the need for retrieval and summarisation. For finance, long-context calls are among the most expensive per request, so measure before making them routine.

Context window vs memory

The context window is what the model sees in this call. Memory is what the system stores outside and decides to bring back in.

Common mistake

My view: pasting everything because it fits. Research such as Lost in the Middle (2023) found models use information at the start and end of long inputs more reliably than the middle. Put the important material first and keep the rest out.

Token · Context engineering · Context rot · Prompt caching · Agent memory

From the archive: A 128K context window needs a concurrency budget

Treat the context window as a budget: important material first, everything else out.

#field-guide#llm#context#tokens

Sources

  1. platform.claude.com/docs/en/build-with-claude/context-windows
  2. arxiv.org/abs/2307.03172
← older drop
AI agent: a model that acts, not just answers
newer drop →
RAG: retrieval-augmented generation explained

related drops

explore all 557 drops →
← back to the archiveday 175