In-Memory Store Concepts

A second running component on the hot path: one more thing to size, secure, upgrade and be paged about, one more way a request can fail, and a store whose entries can be gone by the next request.

An in-memory store guarantees every read it serves is a memory read; a disk engine usually gets one. The gap shows in the tail - a miss costs the engine a device access.

Every entry must be reconstructible from a system of record that still holds what it was made from. No store enforces that — it is a promise the design makes, and the caller must stay correct when an entry is absent.

An in-memory store is rented for four jobs: a derived copy of data a system of record still holds, the sole home for ephemeral state, a coordination point, and a transient transport. Only the first is a cache.

Three facts decide it: whether the entries are a derived copy with a system of record behind them, whether the read path has a route to that source, and whether the source can absorb the tier's full read rate.

On a shared flat keyspace the key string is the entire addressing model, so it must say whose it is: an owning prefix, then the entity, then the identifier, and a format version where the value's shape may change.

A second entry: an application-maintained index keyed by the email address, holding the account's key. The application writes, updates and repairs it, and nothing in the store's base data model builds such a lookup or notices when it is wrong.

A whole-keyspace listing costs the distinct-key count, not the number of matches, and the reply is assembled whole before anything is sent. Where the store executes one operation at a time, every waiting caller pays it.

Multiply the factors out to a distinct-key count, then price each entry as key bytes plus value bytes and multiply again. Compare the result with the tier's budget, and identify which factor is unbounded before arguing about the bytes.

Every operation on that entry is proportional to its whole size: a read ships all of it, any change rewrites all of it, and removing it is real work that someone has to pay for.

Writing one field sends only that field and skips the read: one round trip instead of two, and two callers changing different fields of the same entry stop overwriting each other.

Server-side arithmetic means the caller never holds the old number: one round trip replaces a read-modify-write round trip, and two concurrent callers each contribute a change instead of one silently overwriting the other.

Changing one field costs a read-modify-write round trip: read the whole value, deserialize it, change the field, serialize it again, write the whole value back. Two operations where a store that can address parts of a value needs one.

The caller supplies a number - the ordering score - with every member, and the store keeps the members arranged by it as they are written. Position then becomes a read: the rank of one member, or a slice by position or by score band.

One member collection: the checks are membership tests the server answers without moving the collection, and the group is one unit of work. Choose separate keys only when a membership must expire on its own.

A fixed deadline is set when the entry is written and never moves, so total life is bounded no matter how busy the entry is. An idle deadline is pushed forward by access, so it removes only entries that have gone quiet.

A reader is served nothing: past its deadline the entry is never handed back. The memory is usually still held — stores free an expired entry when something touches it, a sweep reaches it, or the room is needed.

Two different events remove entries. Expiry is removal the application asked for by attaching a lifetime; eviction is removal the store forced on it to free room. Only the second can take an entry early.

A lifetime attached at write makes cleanup the store's obligation: it stops serving the entry at the deadline even if the writing process crashes, is redeployed, or never reaches its delete path. Application-side deletion survives none of that.

Usually the caller, not the store. Many stores do not move a deadline on a read, so an idle deadline is an extra write on every access, turning a read-heavy path into a write-heavy one.

Memory exhaustion has three configured endings: the store refuses writes while reads still work, it removes entries to make room, or the operating system kills the process. Which one you get was a choice, usually a default nobody made deliberately.

An entry also costs a slot in the store's lookup structure, a small header of lengths and pointers, a lifetime field or record if it carries one, and whatever the allocator rounds each of its several allocations up to.

Size for the working set — the entries actually touched in a window — not total data. A database holds the rows, so an absent entry costs one extra read, and memory bought for untouched entries is never read.

Many in-memory stores keep two representations of one value: a compact form that packs elements together, and a general form with a lookup structure per element. Crossing the promotion threshold swaps them, so memory steps rather than climbs.

Because the fixed per-entry cost is charged 10 million times. Data size over entry count is 240 bytes, of which 80 is payload and 160 is lookup slot, header, key bytes and allocator rounding. Overhead scales with count, not bytes.

It promises only that no other caller's operation is applied between the four. It is not a database transaction: there is no rollback, so when the third step fails, the first two stay applied.

The second write lands whole and the first caller's addition is gone. An entry that held 100 holds 110, not 120 — and nothing is reported: both callers were told their write succeeded. That default outcome is last-writer-wins.

Another caller changed that entry between the read and the write, so the store refused it instead of overwriting. The caller must re-read the entry, recompute the change from the new value, and write with the new token.

Per-operation atomicity covers one operation, not a caller's sequence. A read and a write are two operations, and other callers' operations run in the gap between them, so the later write lands whole and the earlier change is gone.

Run-to-completion executes one operation fully before starting the next, so nothing interleaves. Per-entry locking lets worker threads run concurrently, each holding the entry it operates on for that operation's duration. Same promise, different consequences.

"Persistence is on" is not a loss window. Ask which posture is in force and the flush policy or copy interval that governs it — those numbers, not the word, say how many seconds of acknowledged writes a crash costs.

Until entries accumulate again, every read lands on the system of record behind the tier. The multiplier is the inverse of the miss rate: at a 95% hit ratio that system briefly sees twenty times its normal read load.

A point-in-time copy holds the keyspace as of the instant it was cut, not when its file finished writing. Writes accepted after the cut are absent, so up to one whole copy interval of acknowledged writes can be missing.

Keep nothing gives an empty store. A periodic whole copy gives the keyspace as of the last copy. A write log gives it replayed to the last flush to disk. Keeping both replays the log on top of a copy.

A write log records every state-changing operation in the order it was accepted. At start the process replays that log from the beginning, applying each entry again, to rebuild the keyspace it held before it stopped.

One operation runs on one node, so it can only name keys that are all in the same partition. Split across partitions, the call is refused, composed by an intervening layer, or has to be split by the caller.

Acknowledgment means one node has the write in memory, nothing more. Until it propagates, the write exists in exactly one place, and if that node is lost the write is simply gone.

A replica holds a copy of the keyspace but is not the address that accepts writes, so writes resume only after some deciding party promotes one of them to primary. That promotion is a deliberate step, and it takes time.

Assignment must be a deterministic function of the key, evaluated identically by every caller. If two callers disagree about where a key belongs, each reads and writes its own copy on a different node and neither sees the other's.

A replica answers from its own copy, which trails the primary by the propagation lag, so a read issued inside that window returns the value the key held before the write. The write is not lost, only not yet visible there.

Connection setup, paid before any work begins: at least one network round trip for the handshake, plus further trips where an encrypted transport or a credential exchange is required. Microseconds of server work end up buried under milliseconds of setup.

Almost all of it went into the network. Each read pays one full crossing, so 500 sequential reads pay 500 crossings. The store's microseconds are far too small to explain the delay; the number of crossings is the cost.

The other one hundred and ninety-nine were executed and their effects stand. Sending a run of operations without waiting for replies buys latency and nothing else: no atomicity, no isolation, no rollback — each operation succeeds or fails on its own.

Both collapse forty round trips into one, and that is all they share. A pipelined run stays forty independent operations with forty replies matched in order; one operation taking many keys is a single operation with a single reply and a single outcome.

A parked caller consumes a connection slot for the entire wait and essentially no server processing time. The store registers the waiter and goes on serving everyone else, so every store-side signal stays flat while the caller's connection is unusable.

Dead counters accumulate. The keyspace grows with the whole history of subjects and periods rather than with the active ones, and on a tier that removes entries under memory pressure those dead keys crowd out live ones.

A lifetime bounds a crash. If the holder dies between claiming and releasing, an entry that only a release deletes blocks the job forever; the lifetime lets the store drop the claim so another worker can take it.

State held in one instance's memory is readable only by that instance, so the next request may land on a stranger. A shared tier makes all instances equivalent, at the price of a round trip per authenticated request.

Nothing from those two seconds. A broadcast reaches only the connections attached at the instant it is sent, and is then forgotten, so the reconnecting recipient starts from the next message — and neither side can tell a gap happened.

A retry arrives while the first attempt is still running, so a record written after the work is absent exactly when it is needed. Claim it first, with a conditional create: a write that succeeds only if the key is absent.

The store's number is server-side execution only, from the server starting the operation to the reply being produced. The caller's number adds waiting for a connection, both network hops and time queued before execution. That gap is where the slowdown lives.

Stored-data size is one total for the whole keyspace, carrying no owner and no breakdown. Naming a holder is a separate measurement: walk the entries, estimate each one's size, and roll those estimates up per key prefix.

Four quantities hide behind one 'memory used' figure: stored-data size (the server's count for its entries), resident footprint (what the operating system sees), the memory ceiling the server enforces, and the host or container limit above it.

That record times execution only: its clock starts once the server begins an operation, so waiting never enters it. Callers can be timing out on pool wait, on queueing before execution, or on the network while every operation the server actually ran was genuinely quick.

Stock settings in this class of store assume a private network. Many servers listen on every interface and several offer no identity step at all, so whatever can route to the address can read, overwrite and empty the keyspace.