What does the DataLoader pattern's per-key cache do inside a single request?
answer
- The loader does more than batch
- Second job: a map from key to result
- The entry appears before the data does
- Same key twice, one backend call
- No expiry, because no time to expire
basics
~20 sIt memoizes by key. The first load of a key starts the backend fetch; every later load of that same key in that request gets the same result with no second fetch. It deduplicates within a request rather than caching across them.
solid answer
~50 sA loader in the DataLoader batching pattern does two separate jobs, and interviewers usually only ask about the first. Batching collects the keys requested in one tick and hands them to the batch function as a list. Memoization is the second job: the loader keeps a map from key to the in-flight or completed result, so the second, tenth and three-hundredth request for `device-8813` all resolve from that one entry. Because the entry is stored as soon as the fetch starts, not when it finishes, two resolvers asking for the same key concurrently share a single fetch rather than racing into two. The effect is that a document which mentions the same driver on eighty trips causes one lookup, and that a deeply nested document does not re-fetch a parent it already walked past. None of this is in the GraphQL specification; it is a convention of the pattern, and the memo map lives and dies with one loader instance.
code
pseudocode · 11 lines// both resolvers run during the same tick of one request
resolve Trip.driver(trip, ctx):
return ctx.driverLoader.load(trip.driverId) // "drv-2291" -> queued, entry created
resolve Vehicle.lastDriver(vehicle, ctx):
return ctx.driverLoader.load(vehicle.lastDriverId) // "drv-2291" -> map hit, nothing queued
// batch function is called once, with the distinct keys only
batchLoadDrivers(keys): // keys == ["drv-2291", "drv-2288", ...]
rows = store.findDriversByIds(keys)
return keys.map(k -> rows[k]) // same length, same order as keysgo deeper
Recall the two jobs and keep them separate: batch distinct keys into one call, and remember each key so a repeat costs nothing. Be able to say the memory lives for one request only.
Explain the mechanics: the map is checked before the queue, the entry holds the in-flight result so concurrent loads share one fetch, and outcomes including failures are what gets remembered.
Show you read traces rather than assume. Be ready to say what memoization bought versus what batching bought on a real document, and why the absence of expiry is a design choice rather than an omission.
Own the framing that this is a convention every ecosystem re-implemented, not a specified feature, so its guarantees are your servers' contract to state and test rather than something a client can portably assume.
## One object, two jobs The DataLoader pattern is usually introduced as a fix for the N+1 problem, and people remember the batching half: a loader collects every key asked for during one tick of execution and calls a batch function once with the whole list. The second half is memoization, and it is a genuinely different mechanism that happens to live in the same object. A loader holds a map from key to result. When you call `load(k)`: * if `k` is not in the map, the loader creates an entry, queues `k` for the next batch, and stores the *pending* result in the map; * if `k` is already in the map, it returns that entry immediately and queues nothing. So the map is consulted first and the batch queue second. Batching bounds how many calls a *set of distinct keys* costs; memoization bounds how many times the *same* key can cost anything at all. A document that touches four hundred distinct devices and mentions each of them nine times ends up with one batched call for four hundred keys, not nine calls, and not thirty-six hundred. ## The entry is the in-flight result, not the value The detail that separates a candidate who has used the pattern from one who has read about it: the map stores the pending result as soon as the first `load` happens, before any data comes back. That gives single-flight behaviour. If two resolvers, running concurrently on different branches of the document, both ask for `driver-2291` while the first fetch is still outstanding, the second gets the same pending result and waits on it. If the entry were only written on completion, the two would each queue the key and you would be back to duplicate work — the exact thing the pattern exists to remove. That also means a failed load is remembered as a failure. Ask for a key whose fetch rejected and, in the common implementations, you get the same rejection back rather than a fresh attempt; retrying requires evicting the key first. ## A worked example on a fleet telematics graph Take a graph over vehicles, devices, drivers, trips and depots. A dashboard document walks vehicle to trips to driver to that driver's other vehicles and onward — one real document in this shape ran 19 levels deep. Two properties of the data make memoization dominant rather than incidental: * the graph is cyclic in practice — a driver reached through a trip is very often a driver you already resolved higher up; * the fan-out is skewed — a depot's 1,447 trips over a week involve 63 drivers, so driver keys repeat roughly twenty-three times each. Without memoization, batching alone still helps: each level's driver ids get fetched in one call per level. With memoization, the second level onward mostly hits keys already in the map, and the number of backend calls stops tracking document depth. Add a trace and you can see it directly: the count of batch-function invocations stays flat as you add levels, while the un-memoized version grows with them. ## What it is not Calling it a cache invites three wrong assumptions, and interviewers probe all three. **It is not a response cache.** It stores loader values keyed by loader key, not GraphQL responses keyed by document. A different document that needs the same driver benefits only if it runs against the same loader instance. **It has no expiry.** There is no TTL, no size bound and no eviction in the basic pattern. Its correctness comes entirely from being short-lived: it is right because nothing has time to change under it, not because anything invalidates it. Give the same instance a long life and it becomes an unbounded map of increasingly stale values. **It is not specified.** Nothing in the GraphQL specification mentions loaders, batching or memoization. The specification does say that a field's execution may be deferred and that field order within a query selection set is not observable, which is what makes the technique possible, but the pattern itself is convention that every ecosystem re-implemented from the same original design. ## Turning it off Most implementations let you disable the memo map while keeping batching. That is the right switch when values are volatile within a single request — a counter you expect to move as mutations run — and it is the wrong switch when someone reaches for it to "fix" a staleness bug that should have been fixed by evicting the one key that changed. ## Why interviewers ask it Because it separates people who bolted a loader on until the query got faster from people who know what the object does. The follow-up is nearly always about lifetime, and a candidate who has described the memo map accurately has already set up the right answer: it is safe precisely because it lives for one request.
- If two resolvers ask for the same key at the same moment, why is there still only one fetch?Because the loader writes the entry when the first `load` is called, not when the data arrives. The entry holds the pending result, so the second caller finds it already present and waits on the same outstanding fetch. An implementation that only recorded completed values would let concurrent callers both queue the key and would lose the single-flight property.
- Does memoization remove the need for batching, or the other way round?Neither. They cut different costs. Memoization removes repeated work on the *same* key; batching removes round trips across *distinct* keys. A document over four hundred distinct devices, each mentioned once, gets nothing from memoization and everything from batching. A document that mentions one device four hundred times is the reverse. Real documents need both.
- What happens on the second load of a key whose first load failed?In the common implementations the failure is memoized like any other outcome, so the second load gets the same error rather than a fresh attempt. That is usually what you want inside one request — it stops one dead key producing dozens of retries — but it means a deliberate retry has to evict the key first. It is implementation convention, not specified behaviour, so confirm it before relying on either reading.
It is the note you pin to a shared kitchen: the first person to want coffee starts the pot and pins the note, and everyone else who walks in reads the note and waits rather than starting a second pot.
saying these in an interview costs you the question
- Thinks a loader only batches and never memoizes
- Calls it a response cache for GraphQL results
- Assumes the memo map has a TTL or size limit
- Says the GraphQL specification defines DataLoader
- Believes concurrent loads of one key cause two fetches
- Turns the memo map off to fix a staleness bug