What do you gain by wrapping MULTI ... EXEC inside a single Redis pipeline, and what does each of the two mechanisms contribute on its own?
answer
- transport batching vs execution batching
- +QUEUED per command = N+2 round trips
- one write: MULTI, body, EXEC = 1 RTT
- atomic per block, not across blocks
- branching on reads needs WATCH or Lua
basics
~20 sThey are orthogonal and compose. MULTI/EXEC contributes indivisible execution: the queued commands run with no other client's command inside them. The pipeline contributes one round trip instead of one per queued command, because each queued command otherwise gets its own +QUEUED reply.
solid answer
~50 sThey solve different problems. `MULTI`/`EXEC` batches **execution**: commands sent after `MULTI` are queued on the server and then executed as one uninterrupted unit at `EXEC`. Pipelining batches **transport**: bytes travel together and replies come back in order. The reason to combine them is that a transaction is chatty. Each queued command gets an immediate `+QUEUED` reply, so a naive transaction of 20 commands costs 22 round trips. Writing `MULTI`, the 20 commands, and `EXEC` in one pipelined write costs one round trip; the client then reads 22 replies, discarding the `+QUEUED` acknowledgements and using the array returned by `EXEC`. This is exactly what client libraries do for a transaction API. Note that the pipeline adds nothing to the guarantees: a pipeline containing three separate `MULTI`/`EXEC` blocks is atomic per block, not across them, and other clients can run between the blocks. If the batch needs to branch on values it just read, a transaction cannot do it and I use a Lua script instead.
code
text · 16 lines# client writes these 4 commands in ONE socket write (pipelined)
MULTI
SET order:9 paid
INCR paid:count
EXEC
# client then reads 4 replies, in order:
+OK <- MULTI accepted
+QUEUED <- SET queued, NOT executed
+QUEUED <- INCR queued, NOT executed
*2 <- EXEC ran the block as one unit
+OK
:17
# a null reply to EXEC (RESP2: *-1) means a WATCHed key changed
# and NOTHING in the block ran - re-read and retrygo deeper
Say that MULTI/EXEC makes the block run as one unit and the pipeline makes it cost one round trip instead of many.
Explain the +QUEUED reply arithmetic, and that the client must read and discard those acknowledgements before the EXEC array.
Add the boundaries: atomic per EXEC only, WATCH for check-then-act, no rollback for runtime errors, and keeping transaction size bounded so EXEC does not stall the loop.
Frame the choice among pipeline, transaction, and script as where the invariant lives and what it costs the shared execution loop, plus the retry policy and observability the WATCH path implies.
## Two independent axes It helps to name the axes explicitly: - **Transport batching (pipelining):** how many round trips the bytes cost. Client-side only. Guarantees ordering of commands and replies on the connection. - **Execution batching (`MULTI`/`EXEC`):** whether the commands run as one indivisible unit. Server-side. Guarantees no other client's command executes inside the block. Every combination is legal. Plain commands: N round trips, no indivisibility. Pipeline alone: 1 round trip, no indivisibility. Transaction without pipelining: N+2 round trips, indivisible. Transaction inside a pipeline: 1 round trip, indivisible. Only the last one is both fast and safe, which is why it is the default shape in mature clients. ## Why a bare transaction is chatty After `MULTI`, Redis does not execute the commands; it validates that they exist and are well-formed enough to queue, then replies `+QUEUED` for each. Only `EXEC` runs the queue and returns an array of replies, one per queued command. So a lockstep client pays a round trip for `MULTI`, one per queued command to collect each `+QUEUED`, and one for `EXEC`. For a 20-command transaction that is 22 RTTs, which at 0.5 ms is 11 ms of pure waiting for work Redis performs in microseconds. Pipelining collapses that to a single write and a single read phase. The client must still account for every reply: `+OK` for `MULTI`, `+QUEUED` per command, then the `EXEC` array. Miscounting desynchronises the connection, which is why you let the driver's transaction API handle it rather than hand-rolling the framing. ## What the combination does and does not guarantee - **Does:** the whole `MULTI`...`EXEC` body executes with nothing interleaved, at the cost of one round trip. - **Does not:** make several transactions in the same pipeline mutually atomic. `MULTI A EXEC MULTI B EXEC` in one pipeline is two independent units; another client can act between them. - **Does not:** protect the *decision* that produced the commands. Values you read before `MULTI` may be stale by `EXEC` time; guarding that requires `WATCH` on the relevant keys so `EXEC` returns a null reply if any changed, or a script. - **Does not:** change transaction error behaviour. Commands rejected at queue time (unknown command, wrong arity) cause `EXEC` to abort the whole transaction; errors that only appear at execution time do not undo the commands that already ran inside the block. That no-rollback behaviour is a property of transactions, not of the pipeline. - **Does not:** bound resources. A transaction of 100,000 queued commands holds all of them in server memory until `EXEC`, and `EXEC` then occupies the command loop for the entire batch, adding latency for everyone. Keep transactions small for the same reasons you keep pipelines moderate. ## When to reach for a script instead A transaction cannot inspect intermediate results: you cannot queue `GET k`, look at the value, and choose the next command, because nothing has executed yet. Any read-decide-write logic therefore becomes either `WATCH` plus retry (optimistic) or a Lua script / Redis Function that runs server-side with the same indivisibility as a transaction plus real control flow. The pipelining consideration disappears there, because a script is one command and one round trip by construction. ## Reading the replies correctly When you do hand-roll it, the reply stream for `MULTI`, `SET a 1`, `INCR b`, `EXEC` is: `+OK`, `+QUEUED`, `+QUEUED`, then a two-element array. If `EXEC` returns a null reply, a `WATCH`ed key changed and **nothing** in the block ran; the correct response is to re-read and retry, not to assume success. Treating the null as an error, or ignoring it, is a common source of silent data bugs. ## What an interviewer is listening for The clean separation: transport versus execution, plus the concrete arithmetic of `+QUEUED` replies that explains why the two are almost always combined. Strong answers also volunteer that the pipeline confers no extra atomicity across transaction boundaries, and that read-then-decide logic belongs in `WATCH` or a script.
- Two MULTI/EXEC blocks are sent in one pipeline. Is the pair atomic?No. Atomicity is per EXEC: each block runs as its own indivisible unit, and another client's commands can execute between the two blocks. Sharing a pipeline only means the bytes travelled together. If both must be one unit, put all the commands in a single transaction or a single script.
- When would you prefer a Lua script over a pipelined MULTI/EXEC?When the logic must branch on values read during the operation, since queued transaction commands cannot see intermediate results. A script executes server-side with the same no-interleaving property, can read, decide, and write in one pass, and costs a single round trip. The tradeoff is that the script occupies the single execution loop for its whole duration, so it must stay short.
saying these in an interview costs you the question
- 'Pipelining and MULTI are the same thing'
- Thinking a pipeline makes several transactions collectively atomic
- Forgetting to consume the +QUEUED replies and desynchronising the connection
- Treating a null EXEC reply as success instead of a WATCH-triggered abort
- Believing MULTI lets you read a value and then choose the next queued command