For an in-memory tier, when is "keep nothing across a restart and make the restart cheap" the honest choice rather than an unmade decision?
answer
- a posture is always in force
- inherited is not chosen
- three conditions, all of them
- cheap restart is the other half
- sometimes the answer is a different home
basics
~20 sIt is honest when everything in the tier is reconstructible, the system of record behind it can absorb a full rebuild, and the team has weighed that against what a keeping posture would cost while serving. Otherwise it is a default nobody chose.
solid answer
~50 sKeeping nothing is a legitimate posture, and for many tiers the right one — but only when three things hold. Everything in the tier must be a reconstructible copy of state that lives elsewhere. The system of record behind the tier must be able to absorb the rebuild that an empty restart causes, at the traffic level you actually run. And someone must have compared that against what a keeping posture would cost you continuously: memory while a copy is written, latency on the write path from a log, and the cost of replacing a grown log. If all three hold, keeping nothing is the simplest correct answer and you should say so out loud. What makes it dishonest is not the choice but its absence: most teams never picked a posture, inherited whatever the component does, and would give the same answer either way.
go deeper
The idea to take away is that keeping nothing can be a deliberate, correct choice. It is only wrong when something in the tier has no other home, or when whatever sits behind the tier cannot handle rebuilding it.
Be ready to name the ongoing costs a keeping posture adds — memory while a copy is written, latency per write from the log, the cost of replacing a grown log — because that is the side of the trade people forget to mention.
Show that you would verify rather than assume: find out what is actually in memory after a restart by causing one in a controlled window, instead of reading a setting and believing it.
This is your question. Own the decision explicitly, state the conditions it rests on, and be willing to conclude that some state does not belong on this tier at all — while naming what enforcing that costs across many services and many releases.
## The posture nobody chose Start with the uncomfortable observation this question is really about. Ask most teams what their in-memory tier keeps across a restart and you will get an answer about the component rather than about a decision — "I think it persists", "there's a file somewhere", "whatever the platform team set up". **A posture is always in force.** If nobody chose it, one was defaulted into, and the defaulted-into posture has the property that nobody has priced it in either currency: not the writes it would lose, not what it costs while the tier is serving. That is what separates "keep nothing" as a decision from "keep nothing" as an accident. They look identical in a configuration listing and are entirely different engineering. ## When keeping nothing is the right call Three conditions, all of which have to hold: 1. **Everything in the tier is reconstructible.** Every entry is a copy of state whose home is elsewhere, or is disposable — a derived result, a computed view, a materialised lookup. Nothing in the tier is the original of anything. 2. **The system of record behind the tier can absorb the rebuild.** An empty restart moves the tier's load onto the system behind it for as long as it takes to repopulate, and the question is whether that system has the headroom at your real traffic level, not at your average one. 3. **Somebody compared the alternative.** Keeping a posture is not free: memory while the live keyspace and a copy diverge, latency added to every write by the log's flush policy, and the periodic cost of replacing an accumulated log with a shorter one. If those costs buy you nothing — because condition one already made the data disposable — then paying them is the unexamined choice, not the safe one. ## "Make the restart cheap" is the other half The sentence is not "keep nothing" alone. Choosing to keep nothing puts an obligation on the restart, and meeting it is what makes the choice defensible: - Keep the working set small enough that repopulating it is a burst the system behind can take, rather than an outage. - Know what an empty tier does to your end-to-end latency, and be able to say whether that is degradation or failure. - Make restarts routine and boring, so that the first empty restart is not the one during an incident. - Be able to state the rebuild cost in the same breath as the choice. If nobody can, condition three has not been met. ## The two ways to get this wrong | Failure | What it sounds like | Why it is wrong | |---|---|---| | Persistence as reflex | "Keeping something is always safer" | It charges the write path or memory forever to protect data that is reconstructible anyway | | Keeping nothing as reflex | "It's a cache, it doesn't matter" | Some tiers hold ephemeral state with no other home, and that state is not reconstructible | | Promotion by persistence | "We keep a log, so it's safe to store it here" | The window shrinks but never closes, so sole-copy state is still exposed | The third row is the one that does real damage, because it is how a volatile tier quietly becomes a system of record without anyone deciding that it should. ## What ruling the tier out actually costs Sometimes the honest conclusion is stronger than choosing a posture: the state in question cannot tolerate any window at all, and therefore does not belong in this tier under any posture. That ruling is cheap to say and expensive to implement. It means finding another home for that state, changing the code that writes it, and — if the pattern is repeated across many services — holding the line every time someone proposes the tier as a convenient place to put something. A principal-level answer names that cost rather than issuing the rule and moving on. ## How to close the answer Say which posture is in force today and how you know. Say whether it was chosen. Then give the three conditions and check them out loud against the workload in front of you. The strongest version ends where this whole subject ends: persistence on this tier bounds a restart, it does not make the tier the only copy of anything, and if what you are protecting cannot survive a window, the answer is not a tighter posture but a different home for the data.
- How would you find out which posture is actually in force across a fleet of tiers?Ask the question as "what is in memory after a restart", not "is persistence on", because the second gets you a configuration answer and the first gets you a behavioural one. Then verify rather than trust: restart a tier deliberately, in a controlled window, and observe what comes back and what the system behind sees. A posture nobody has watched take effect is a belief, not a fact.
- A team wants to add a keeping posture purely to reduce incident severity. What do you ask them?What the tier holds, and whether any of it is irreplaceable. If everything is reconstructible, the posture buys a shorter and smaller rebuild — a real benefit, worth pricing against the write-path and memory costs it adds permanently. If something is irreplaceable, the posture is being used as a substitute for a decision about where that state lives, and it will not hold, because the window never closes.
- Is there a case where keeping nothing is better even though the data would benefit from being kept?Yes, when the ongoing cost falls on a path that cannot afford it. A tier on a hot write path may not be able to absorb a per-write flush, and a tier with a high write rate pays a large divergence cost every time a copy is taken. If restarts are rare and the rebuild is survivable, spending nothing while serving and accepting a costlier restart is a coherent trade.
saying these in an interview costs you the question
- Treats keeping something as always safer than keeping nothing, whatever it costs
- Assumes an inherited setting was chosen deliberately by someone earlier
- Says keeping nothing removes the need to plan for restarts
- Believes a keeping posture makes the tier a valid system of record
- Claims a posture is free once it has been switched on
- Rules state out of the tier without naming what that ruling costs