← back to the archiveCover illustration for “Prompt caching: paying less for repeated context”
TERM8mo ago·Published · Updated ·by Andy Padia

Prompt caching: paying less for repeated context

In short: Prompt caching lets a model provider reuse its processing of a repeated prompt prefix, so later requests cost less and start sooner.

Prompt caching is a provider feature that stores the processed form of a prompt's opening section, its prefix, so a repeated prefix is not reprocessed. Cached input tokens cost less and the response starts sooner: Anthropic's documentation, read in November 2026, prices cache reads at a tenth of the base input rate for most models. It suits apps that resend the same long context on every call.

Prompt caching in plain words

Re-reading the same 50-page manual before every question is slow. Caching is keeping your notes from the first read and reusing them.

How it works

The provider keeps the work it did on a prompt prefix for a limited time. A new request with an identical prefix reuses that work, and only the new part is processed at the full rate.

OpenAI's prompt caching guide, read in November 2026, turns caching on by default for supported models and says reuse requires the entire prefix to match. Anthropic's documentation offers automatic caching or explicit cache_control breakpoints, keeps entries for 5 minutes by default or 1 hour on request, and prices 5-minute cache writes at a quarter above the base input rate.

Changing even one token early in the prefix breaks the match.

Example

As an illustration, a support bot sends the same system prompt and a 6,000-token policy manual plus a short question on every call. Put the manual first and the question last, mark the manual as cacheable, and every call inside the cache lifetime after the first pays the reduced rate on the manual and the full rate only on the question.

Where you'll meet it

  • Model APIs from Anthropic, OpenAI and other providers
  • Coding agents, which resend large system prompts and file context each turn
  • Chat over long documents
  • Agent loops with long, stable tool definitions

What it changes for you

For developers, prompt order becomes a cost decision: stable content first, variable content last. For finance, caching changes the unit cost of long-context features, so check cached-token counts in usage reports. For architects, it shifts the trade-off between sending a long context and retrieving only what is needed.

Prompt caching vs response caching

Prompt caching reuses the processing of an input prefix, and the model still generates a fresh answer. Response caching stores a previous answer and returns it for an identical question.

Common mistake

My view: putting a timestamp, user name or random ID at the top of the system prompt. It changes every request, so nothing below it ever caches. Move variable details to the end.

KV cache · Token pricing · Context window · Cost per task · Time to first token

Keep the stable part of a prompt first and the changing part last, or nothing below the change will cache.

#field-guide#llm#cost#inference#optimization

Sources

  1. platform.claude.com/docs/en/build-with-claude/prompt-caching
  2. developers.openai.com/api/docs/guides/prompt-caching
← older drop
Hybrid search: combining keyword and vector search
newer drop →
Token pricing: how AI model APIs charge you

related drops

explore all 557 drops →
← back to the archiveday 175