Why would a caller ask a store to hold its request until an entry appears instead of re-asking every hundred milliseconds?
answer
- two ways to wait, different bills
- trips and interval, or a held slot
- the interval is latency you chose
- parked means one trip, one slot
- not every store offers the call
basics
~20 sParking trades network trips for a held connection: one call instead of one per ask, and the caller learns of the entry a round trip after it is written rather than an interval later. Not every store offers it.
solid answer
~50 sBoth shapes are honest; they spend different resources. Re-asking on a schedule costs one round trip per ask, and the interval is added latency — an entry written just after an ask is invisible for nearly a full interval — but the connection goes back to the pool between asks, so a thousand waiters can share a small pool. A wait-with-deadline call costs one trip for the whole wait and wakes the caller a round-trip time after the value is written, but the connection is held for the entire wait while the server does no work for it. So: park when waiters are few and latency matters; re-ask when waiters are many and the pool is shared. A large part of this class offers no such call at all, in which case the interval is your whole design.
go deeper
Recall that there are two ways to wait for a value that is not there yet: ask again on a schedule, or ask once and let the server hold the request until a deadline. Know that the second one is not offered by every store.
Explain the bill each shape presents: trips plus interval latency for re-asking, one trip plus a connection slot held for the whole wait for parking. Be able to do the trip arithmetic for a concrete interval and wait length.
Show the judgment: count the waiters against the pool before choosing to park, and state the placement, because the round-trip time is what the interval is trading against. Say out loud that the store's own signals will not show a waiting population.
Set the rule for the platform: which workloads may park at all, what bounds the wait deadline, and where waiting callers get connections from. Treat the choice as a capacity decision about slots, not a micro-optimisation about latency.
## The situation A value is not in the store yet. Some other process — a producer, a dispatcher, another request — will write it, and this caller has to act when it arrives. The tier answers in microseconds, so the interesting cost is never the store's work. It is how many times the caller crosses the network, and how long a connection is held while nothing happens. There are exactly two shapes for this, and stores in this class disagree about whether the second one exists. ## Re-asking on a schedule The caller reads the key, finds nothing, sleeps for an interval, and reads again. Each read is one round trip. The **server's service time** — what the store spends executing — is microseconds; the **caller's wall time** — what the caller measures end to end — is dominated by the **round-trip time**. Co-located that is tens of microseconds, one zone away a fraction of a millisecond, a different region and the design is already wrong. Two properties follow, and together they are the whole trade-off: - **The connection is free between asks.** A slot is taken for one round trip and returned to the pool. A thousand waiters sharing twenty connections is workable. - **The interval is added latency.** A value written just after an ask is not seen until the next one. The average added delay is half the interval; the worst case is all of it. Halving the interval halves that delay and doubles the trips: thirty seconds of waiting at a hundred-millisecond interval is three hundred trips per waiter, and two hundred waiters make sixty thousand calls the store had no reason to answer. ## Parking with a deadline Where the store offers it, the caller issues one call naming the key and a **server-side wait deadline**. The server executes nothing: it records that this connection is waiting on that key and goes back to serving everybody else. When the value is written the parked caller is answered; if the deadline passes first, the call comes back empty. - **One trip for the whole wait**, not one per interval. - **The delay after the write is one round-trip time**, not up to an interval. - **The connection is held for the entire wait.** That slot is unavailable to anything else in the process for as long as the caller is parked — even though the store is doing no work for it. This is the cost that surprises people, because no store-side signal shows it. | | Re-asking on a schedule | Parking with a deadline | |---|---|---| | Trips for a thirty-second wait | one per interval | one | | Delay after the value is written | up to one interval | one round-trip time | | Connection slot held | one round trip per ask | the whole wait | | Server processing time | microseconds per ask | essentially none | | Offered by every store of this class | yes | no | ## Choosing between them 1. **Few waiters, slots to spare, latency matters** — park. One call, wakeup a trip after the write, and the slots you spend are affordable. 2. **Many waiters on one shared pool** — re-ask. Each parked caller holds a slot, and a waiting population near the size of the pool will starve everything else in the process that shares it. 3. **The value is usually already there** — re-ask. The first read answers, and you never pay the interval at all. 4. **The store has no such call** — re-ask, and pick the interval deliberately. It is your latency budget, written down. ## What the capability is not Two boundaries are worth stating, because both get assumed. - It is **not universal**. A large part of this class answers only single-key reads and writes of opaque values. There is nothing to park on, and no client configuration adds one. Ask which store before designing around the call. - It is **not a message broker**. A parked caller gets no acknowledgement, no redelivery if it dies while handling what it received, and no replay of anything it missed while it was disconnected. Those are properties of durable messaging systems. Whether a particular store also offers a separate structure with acknowledgement semantics varies store by store, and that is a different mechanism with its own rules. Both designs are defensible. The mistake is choosing one without saying which resource you decided to spend: network trips and latency granularity, or connection slots.
- What exactly does the polling interval cost that parking does not?Two things. Latency: a value written just after an ask is invisible for the rest of the interval, so the average added delay is half the interval. And trips: each ask is a full round trip whose server work is microseconds, so the network, not the store, pays for every one of them.
- When is re-asking on a schedule the better choice even where parking exists?When the waiting population is large relative to the connection pool, since every parked caller holds a slot for its whole wait; when the client library cannot give that one call its own deadline; and when the value is usually present already, so the first read answers and the interval never costs anything.
saying these in an interview costs you the question
- Says parking is free because the server does no work during it.
- Assumes every in-memory store offers a wait-with-deadline call.
- Treats a parked caller as a queue consumer with acknowledgement and redelivery.
- Shrinks the polling interval without counting the trips it adds.
- Picks parking for thousands of waiters sharing one small pool.