skip to content

With the keyspace split across partitions and no way to co-locate, how do you rework an operation over four related keys?

level: middleimportance: must knowfreq 62%

answer

  1. must they agree, or just travel together
  2. split the call, or one entry
  3. four calls means four moments
  4. one key is one partition
  5. decide what a torn read does

basics

~20 s

Three reworks are honest: one call per key composed in the caller, folding the four values into one entry under one key, or accepting the operation is no longer one unit and saying what a reader does with a mixed view.

solid answer

~50 s

There are three real options once co-location is off the table. Issue one call per key and compose in the caller: placement stays even and every store supports it, but the four values now come from four moments, so another writer can leave you with a torn view. Fold the four values into a single entry under one key: one key is one partition, so whatever indivisibility the store gives one key now covers the group - at the cost of a larger entry, that group's traffic on one node, and, on stores that only hand back opaque bytes, a full read and rewrite for any change. Or accept that it is no longer one unit and design the reader for it. The choice turns on whether the four values must agree with each other or merely travel together.

go deeper

for a junior

Know that the fix is either putting the related keys together or taking the operation apart. Being able to name both directions is enough at this stage; the trade-offs come later.

for a middle

Lay out all three reworks and what each costs, and start from the question of whether the four values must agree or merely travel together. That question, not the mechanics, is what the interviewer is listening for.

for a senior

Show the consequences the rework creates: four calls means four moments and a torn view; one entry means one node carries that group's bytes and traffic. Then say what the reader does when it sees the group half-changed.

for a principal

Treat the choice as a contract change, not a refactor. Decide once, per class of state, whether this tier is allowed to hold values that must agree with each other, and make that the rule new services are reviewed against.

## The situation The design was written against a single node: one operation read four related values, changed them, and wrote them back, and nothing about the tier made that suspicious. The tier now has each node holding a share of the keyspace, so each of the four keys is in whichever partition its name lands in, and the operation cannot run as one call. Co-location is not available to you: either the routing shape honours no marker in the key name, or the group is too large to place as a unit, or the keys are written by services you do not own and cannot rename this quarter. What is left is reworking the operation itself. Start by asking the question that decides everything else: **must the four values agree with each other, or do they merely travel together?** A quota counter and a last-seen timestamp travel together; a claim and the record of who holds it must agree. Only the second case actually needs the indivisibility you are about to lose. ## Option 1 — one call per key, composed in the caller Issue four single-key calls and assemble the result in the application. - **Keeps:** placement stays even, because each key is still spread by its own name; the code works on every store in this class, whatever its routing shape; nothing needs renaming. - **Gives up:** the four values are read at four different moments. If another caller changes the fourth key after you read the first, you compose a view that never existed. It also multiplies calls, and what those calls cost and how they are collapsed is a separate concern from this one. - **Pick it when:** the values are independent, or a slightly torn view is detectable and harmless. ## Option 2 — one entry holds the whole group Store the four values as one structured value under one key. - **Keeps:** one key is one partition by construction, so whatever the store makes indivisible for a single key now covers the whole group, and the call count drops to one. - **Gives up:** the entry grows, and one entry is one key on one node, so all of that group's traffic and all of its bytes land in one place. On stores where the server only hands back opaque bytes, changing one field means reading the whole value and writing the whole value, which is both more traffic and a window in which two changers can overwrite each other. Where the server interprets the value, part of it can be changed in place and that window is much smaller. - **Pick it when:** the four values are genuinely one thing that was only ever split because four keys were easy to type. ## Option 3 — stop pretending it is one unit The third option is not a failure to solve the problem; it is frequently the right answer, and stating it plainly is what separates a considered design from an accident. 1. Make a mixed view **detectable**: give the group a version or a generation marker written last, so a reader that sees an old marker knows the rest may be mid-change. 2. Make the group **re-derivable**: if the values can be recomputed from a system of record, a reader that sees them inconsistent can discard and rebuild rather than act. 3. Make the order **carry the meaning**: write the value that other callers key their decision on last, so an observer either sees the whole change or sees none of the part that matters. What you must not do is leave it unstated. An operation that used to be indivisible and now is not, with nobody having decided what that means, is an incident waiting for the first concurrent caller. ## Choosing between them | Rework | Keeps | Gives up | Right when | |---|---|---|---| | One call per key | Even placement, works anywhere | A single-instant view of the group | The values are independent | | One entry for the group | Indivisibility, one call | Entry size, one node's share of load, whole-value rewrites on byte-only stores | The values are one thing | | Accept a mixed view | Simplicity, no migration | The guarantee itself | A mixed view is detectable or cheap to repair | ## What varies between stores Three of the facts above are not universal, and an answer that states them flatly is describing one product. Whether the server interprets the stored value at all decides how expensive option 2 is. Whether any co-location mechanism exists decides whether you were forced here in the first place. And whether more than one key can ever be changed as one unit varies as well — some stores in this class offer no multi-key change at all, in which case option 3 was always the situation and the split merely made it visible. ## The shape of the two structural reworks, written against an invented store interface. The loop is the point: four separate calls produce four separately-timed values, which is exactly the property the original call had and the rework does not ``` # before - one call, several keys, one address space assumed values = store.readMany([k1, k2, k3, k4]) # after (a) - one call per key, composed by the caller values = [] for k in [k1, k2, k3, k4]: values.append(store.readOne(k)) # each value is from a different moment: another caller may # have changed k4 after k1 was read # after (b) - one entry holds the whole group group = store.readOne(groupKey) # one key, therefore one partition ```

  • How do you choose between composing in the caller and folding the group into one entry?
    By whether the values must agree. If a reader can act on one of them without the others, keep them separate and compose. If acting on one while another is stale is a bug, fold them into one entry and pay the size and single-node cost. Access pattern breaks the tie: values always read together want one entry.
  • What goes wrong if half the callers are reworked before the split and half are not?
    The unreworked half meets the constraint in production, on whichever path is least exercised, and fails there first. Worse, if an intervening layer composes multi-key calls instead of refusing them, the unreworked half keeps working and silently reads torn groups, which is harder to notice than an error.

saying these in an interview costs you the question

  • Assumes composing four separate reads still gives a single-instant view
  • Thinks folding values into one entry is free of placement consequences
  • Believes a group of steps submitted together can span partitions
  • Says the store will coordinate the nodes if you ask it to
  • Treats accepting a mixed view as never a legitimate answer
  • Assumes every store can change part of a stored value in place