HTTP cache keys must include the view
In short: Retrieval caches need representation matching, access isolation and freshness controls because a URL alone cannot identify the document view.
↗IETF — HTTP caching and the Vary fieldRFC 9111, published in June 2022, explains why a cached HTTP response cannot always be selected by URL alone. A response's Vary field names request headers that influenced the representation, and cache reuse must account for them.
HTTP cache keys must include the view. For an agent retrieval tool, my broader recommendation is to preserve the representation and access context that gave a document its meaning. “We fetched this URL already” is an incomplete explanation when the response changes with language, encoding or who is allowed to see it.
What Vary contributes to cache selection
The HTTP caching specification describes matching request fields nominated by Vary. The companion HTTP semantics reference explains how that field expands the information used to select a stored response.
These are useful documents for engineers building retrieval wrappers, especially when a generic URL-to-text dictionary quietly replaces a real HTTP cache. The dictionary may discard headers and freshness information precisely because the text extraction step looks successful.
Preserve the retrieval context
For a hypothetical internal research assistant, I would separate public retrieval caching from authenticated document access. Access context must remain isolated even when two requests use the same URL. Vary is not an authorisation system, and a missing Vary header does not grant permission to share a private response.
I would also retain the canonical source URL, response timestamp, relevant validators and extraction version beside the stored text. The extraction version is my application-level addition: changing the parser can change what evidence the agent sees without any change at the origin.
The test I would request is simple: fetch two deliberately different representations of one resource and verify that reuse selects the intended one. Add a separate access-boundary test using controlled accounts. These are proposed fixtures, not tests performed against a private service.
The limitation is equally important: a well-keyed cache can still contain stale or incorrect content. Correct cache selection does not certify the source's truth.
What's in it for you
- Inspect what your URL-to-text cache throws away.
- Keep access isolation separate from representation matching and freshness.
Cache the document view you actually obtained, not just the address you typed.
