skip to content

Redis Lua scripts historically had strict determinism rules — no random values, no reliance on the clock, sorted output for unordered commands. Why did those rules exist, and what changed once Redis started replicating script effects instead of the script itself?

level: seniorimportance: should knowfreq 28%

answer

  1. Old: EVAL shipped verbatim, replica re-ran it
  2. Divergence risk ⇒ random/TIME restricted, replies sorted, PRNG seeded
  3. 3.2 redis.replicate_commands(), default in 5.0
  4. Now: only the write commands propagate, in MULTI/EXEC
  5. Nondeterminism safe; keys/atomicity/no-rollback unchanged

basics

~20 s

Originally Redis shipped the script itself to replicas and the AOF, so both sides had to produce identical results — any randomness or clock use would diverge them. Modern Redis replicates the script's resulting write commands instead, so nondeterminism is safe, though determinism still aids reasoning.

solid answer

~50 s

**Verbatim replication (the old default):** the primary sent the `EVAL` to replicas and to the AOF, and each re-executed it. Identical results were therefore mandatory. Redis enforced this: `TIME`, `SRANDMEMBER`, `RANDOMKEY`, `SPOP` and friends were restricted or required `redis.replicate_commands()`, `math.random` was seeded identically per execution, and commands with unspecified ordering (`SMEMBERS`, `KEYS`, `HGETALL`) had their output sorted so a script could not branch on arbitrary order. Writing after a random read was rejected. **Effects replication:** since Redis 3.2 (opt-in via `redis.replicate_commands()`) and the default since 5.0, the primary executes the script locally and propagates only the **write commands it actually performed**, wrapped in a MULTI/EXEC. Replicas never run the Lua. Nondeterminism is then harmless — a random or time-based value chosen on the primary is replicated as a literal. Determinism is no longer a correctness rule, but keeping scripts predictable still helps debugging and reasoning.

code

lua · 7 lines
lua
-- reads a random member, then writes: rejected in the pre-5.0 verbatim mode
local winner = redis.call('SRANDMEMBER', KEYS[1])
redis.call('SET', KEYS[2], winner)
return winner

-- under effects replication the primary picks the winner once and
-- propagates the literal: SET KEYS[2] "<that member>"

go deeper

for a junior

Know that modern Redis sends the script's resulting write commands to replicas rather than the script itself, so scripts do not have to be deterministic.

for a middle

Explain the old verbatim model and why it forced restrictions on TIME, random commands and unordered replies, and name effects replication as the change.

for a senior

Give the version story (3.2 opt-in, 5.0 default), describe the MULTI/EXEC-wrapped effects, note the propagation-cost benefit, and say determinism is now a testability habit rather than a rule.

for a principal

Discuss the design shift — moving the point of decision to the primary — and its consequences for reasoning about replica state, incident reproducibility and time-dependent data becoming durable input.

## Why determinism ever mattered When scripting arrived in Redis 2.6, replication and the AOF worked by **verbatim propagation**: the primary forwarded the `EVAL` command — the script body plus its keys and arguments — to its replicas and wrote it to the append-only file. Each replica then executed the script itself, and AOF replay executed it again at load time. That design has an obvious requirement: the script must produce the **same effects everywhere**. If the primary's run wrote `x = 0.42` from `math.random()` and the replica's run wrote `x = 0.91`, the datasets have silently diverged, and nothing in the system would ever notice. The same for `TIME` (different clocks, different instants), for `SPOP`/`SRANDMEMBER`/`RANDOMKEY` (arbitrary element selection), and for any command whose reply order is unspecified. ## How Redis enforced it Redis defended the invariant with several mechanisms: - **Command classification.** Commands flagged as random — `SPOP`, `SRANDMEMBER`, `RANDOMKEY`, `TIME`, `LASTSAVE` and others — were restricted inside scripts. In particular, calling one and *then* performing a write was rejected with an error about writing against a non-deterministic command. - **A seeded PRNG.** Lua's `math.random` was replaced with a deterministic generator seeded identically for every script execution, so the same script produced the same sequence on primary and replica. `redis.replicate_commands()` and later effects replication removed the need for this; you could also change the seed explicitly with `math.randomseed`, at the cost of the guarantee. - **Sorted output.** Replies from commands with no defined ordering (`SMEMBERS`, `SINTER`, `KEYS`, `HKEYS`, …) were sorted before being handed to Lua, so a script could not branch on the arbitrary internal iteration order of a hash table, which differs between nodes. These rules were a persistent source of surprise: a script that worked in isolation would fail as soon as you added a write after a random read. ## The shift to effects replication Redis 3.2 introduced `redis.replicate_commands()`, a call a script could make to switch itself from verbatim to **effects replication**. Redis 5.0 made effects replication the **default and only** mode, and the explicit call became a no-op returning true. Under effects replication the flow is: 1. The primary executes the script locally. 2. Redis records the **write commands the script actually issued** — the concrete `SET k v`, `INCRBY k 3`, `ZADD …` with their final literal arguments. 3. Those commands are propagated to replicas and to the AOF, wrapped in `MULTI`/`EXEC` so the batch is applied atomically on the other side. 4. Replicas never see the Lua at all. The consequences are large: - **Nondeterminism becomes safe.** A value from `TIME`, from `math.random`, from `SPOP` is chosen once, on the primary, and travels as a literal. Divergence is impossible because there is no second execution. - **Restrictions relax.** Writing after a random read is fine; the sorting behaviour and the seeded PRNG lose their reason to exist. - **Propagation is proportional to effects, not to work.** A script that reads a million elements and writes one key propagates one command — a real efficiency gain over shipping a script that replicas must re-run. - **Read-only scripts propagate nothing.** ## What is still true - **Keys must still be declared.** Effects replication changes propagation, not routing or ACL checks. - **Atomicity and blocking are unchanged.** The script still occupies the single command-processing thread and still cannot be rolled back. - **Determinism is still a good habit.** A script whose output depends on the wall clock is hard to test, hard to reason about during an incident, and produces replica data you cannot reproduce from the inputs. Determinism stopped being a *correctness* requirement; it did not stop being good engineering. - **The clock is still per-node.** `TIME` inside a script returns the primary's clock. If your logic depends on time, that dependency now propagates as data — which is usually what you want, but it does mean the replicated value reflects the primary's clock skew. ## Interview framing The crisp answer is: determinism was required because replicas re-executed the script; once Redis switched to replicating effects, the primary is the single point of decision and nondeterminism cannot cause divergence. If you are asked which version, say `redis.replicate_commands()` arrived in 3.2 and effects replication became the default in 5.0. If asked whether you would still write nondeterministic scripts freely, say yes for correctness, but that you keep them predictable for testability and incident reasoning.

  • Under effects replication, how much traffic does a script that scans a large collection but writes one key generate towards replicas?
    Only the single write command it performed, wrapped in MULTI/EXEC. The reads are local work on the primary and are never propagated. That is a significant improvement over verbatim replication, where the replica would have to re-execute the whole scan to arrive at the same state.
  • Is it now safe to use TIME inside a script that writes?
    Yes. The primary reads its own clock, computes the value once and propagates the resulting write with that literal value, so replicas and the AOF cannot diverge. The remaining caveat is semantic rather than mechanical: the stored value reflects the primary's clock, including any skew, and it becomes part of the data rather than something recomputed at replay time.
  • Does effects replication change anything about how a script must declare its keys?
    No. Key declaration exists for cluster slot routing and ACL key checks, which happen before the script runs and are independent of how the results are propagated. Every key the script reads or writes must still arrive through the declared key arguments, and in Cluster they must all hash to the same slot.

Verbatim replication is faxing the recipe and hoping every kitchen cooks the identical dish; effects replication is cooking once and shipping the finished dish.

saying these in an interview costs you the question

  • Claiming replicas still execute the Lua script in current Redis versions
  • Saying scripts must be deterministic today or they will corrupt replicas
  • Thinking effects replication also removes the need to declare keys
  • Believing math.random is still seeded identically per execution as a correctness mechanism
  • Assuming the AOF stores the script body rather than its resulting commands

context

resolving links…