← back to the archiveCover illustration for “Context rot: when long context makes models worse”
TERM8mo ago·Published · Updated ·by Andy Padia

Context rot: when long context makes models worse

In short: Context rot is the drop in an LLM's accuracy as its context fills with long or irrelevant material, and the reason long agent sessions need pruning.

Context rot is an informal name for the way a language model's accuracy degrades as its context window fills up, especially with long, repetitive or irrelevant material. Chroma's Context Rot report from July 2025 tested 18 models and found that performance grows less reliable as input length grows. It is one reason long agent sessions drift.

Context rot in plain words

Picture a meeting that has run three hours. Everything said is technically in the room, but nobody remembers the decision made in hour one.

How it works

Models do not use every part of a long input equally. Liu and colleagues showed in July 2023 that models answer best when the relevant information sits near the beginning or end of the input and noticeably worse when it sits in the middle, even for models built for long contexts. Chroma's report adds that distractors hurt more as inputs get longer.

In agents, old tool outputs, failed attempts and superseded plans pile up and keep influencing the next step. Anthropic's context engineering guide from September 2025 uses the same term and describes three responses: compaction, notes kept outside the window, and sub-agents that work in clean contexts.

Example

As an illustration, a coding agent's first instruction was "never edit the generated folder". Two hours and many large test logs later, it edits a generated file. The rule is still in context, but everything since outweighs it.

Where you'll meet it

  • Long coding-agent sessions and chat threads
  • RAG systems that stuff many chunks into one prompt
  • Research on long-context benchmarks such as needle-in-a-haystack tests
  • Harness features named compaction, summarisation or clearing context

What it changes for you

For developers, start fresh sessions for new tasks, clear or compact regularly, and move exploration into subagents. For architects, retrieve fewer, better chunks instead of more. For teams, assume quality drops late in long runs and put checks there.

Context rot vs hallucination

Hallucination is the model producing unsupported content. Context rot is a cause: degraded use of what is actually in the context can lead to errors, including hallucinations.

Common mistake

My view: trusting the advertised context size as usable size. Treat the window as a ceiling you rarely want to reach, and test your own task at the lengths you plan to use.

Context window · Context engineering · Subagent · Reranking · Agent memory

From the archive: Context compaction needs a recovery test

The advertised context window is a ceiling, so test your own task at the lengths you plan to run.

#field-guide#context#llm#reliability#agents

Sources

  1. trychroma.com/research/context-rot
  2. arxiv.org/abs/2307.03172
  3. anthropic.com/engineering/effective-context-engineering-for-ai-agents
← older drop
KV cache: the memory that keeps LLM generation fast
newer drop →
Quantization: shrinking a model by using fewer bits

related drops

explore all 557 drops →
← back to the archiveday 175