Cache awaited values in sequential Python batches

The first lookup worked. The second lookup, with the same key, raised RuntimeError. In a local Python 3.14.6 check on 12 October 2026, putting lru_cache on an async def function cached something—but not the completed value the caller wanted.

Cache awaited values in sequential Python batches, rather than caching the coroutine returned by the loader. For a small batch, my default is a dictionary owned by that one batch. The useful guarantee is modest: repeated keys reuse successful results without carrying cache state into the next job.

Why lru_cache retains the wrong object

Python's functools reference explicitly warns against using this cache for async functions. Calling an async function produces a coroutine object. A synchronous decorator can retain that object before anything awaits its result.

The Python data model specifies that awaiting a coroutine more than once raises RuntimeError. So the first call can look healthy while the next matching call retrieves an already-consumed coroutine. A cold-cache test misses the defect by design.

Don't judge this wrapper from its first successful result. Exercise the same key twice, and inspect what the cache actually stores. A hit counter is not proof that the returned object remains usable.

Cache awaited values inside the batch loop

Save this as batch_cache.py and run python3 batch_cache.py. It uses synthetic strings, no packages or network calls. The loop is deliberately sequential; each load completes before the next key is considered.

import asyncio

async def load_batch(keys, load):
    # Finite, sequential batch; values must be safe to share.
    values = {}
    output = []
    for key in keys:
        if key not in values:
            values[key] = await load(key)
        output.append(values[key])
    return output

async def main():
    calls = []

    async def load(key):
        calls.append(key)
        await asyncio.sleep(0)
        return None if key == "missing" else key.upper()

    result = await load_batch(["a", "missing", "a", "missing"], load)
    assert result == ["A", None, "A", None]
    assert calls == ["a", "missing"]
    print("4 outputs, 2 loader calls; None reused correctly")

if __name__ == "__main__":
    asyncio.run(main())

The assignment happens after await load(key) succeeds. The dictionary holds the returned value, not the coroutine. Membership checking matters too: None, False and an empty string can be legitimate results. A truthiness check would confuse a cached empty value with a missing entry.

If loading raises, the assignment does not complete and the batch fails. There is no hidden retry or conversion to a success-shaped placeholder. A new invocation starts with an empty dictionary, including after a failed invocation.

What the local Python checks established

The exact snippet produced four ordered outputs with two loader calls, including reuse of None. Additional checks covered empty input, distinct keys, output order and a fresh load in a new batch. Synthetic failure and cancellation cases propagated instead of processing later keys.

These are functional observations, not a latency or cost benchmark. The negative test separately confirmed that an lru_cache-decorated async function succeeded once and failed on its second await. No database, model endpoint or production agent was exercised.

My judgement for a document-processing batch

For a hypothetical document batch that repeatedly reads the same stable metadata, I would start with this short-lived cache before introducing a service-wide one. Its lifetime is visible in the function. There is less invalidation policy to explain because the next batch reloads everything.

That recommendation depends on the key identifying the complete lookup context. If tenant, revision or access scope changes the result, it belongs in the key or in a separate batch. Caching by document ID alone is not an authorisation design.

There are limits worth keeping visible. Keys must be hashable; results must be safe to share because repeated outputs reference the same object. Memory grows with the batch and its unique values. Earlier loader side effects are not rolled back when a later load fails.

Do not turn the loop into concurrent tasks and assume it now coalesces simultaneous misses. That is a different requirement needing explicit in-flight coordination, cancellation rules and tests. This helper offers neither TTL nor a bounded, process-wide LRU cache. Its small scope is the point.

What's in it for you

• Test the second lookup, not just the first successful await.
• Reuse completed results within a finite batch without retaining coroutine objects across calls.

Cache the answer you can reuse, and make the lifetime of that answer visible.

Sources

• Python — functools, Python 3.14 documentation, accessed 12 October 2026.
• Python — Data model, coroutine objects, Python 3.14 documentation, accessed 12 October 2026.