skip to content

Rows must be created on several database shards, in more than one region, and by offline clients that assign an identifier before the row ever reaches a database. How would you choose an identifier strategy for that, and what are the trade-offs?

level: principalimportance: nice to knowfreq 30%

answer

  1. Where is the id minted, and can you coordinate
  2. 64-bit compact vs 128-bit decentralised
  3. Snowflake: node id assignment + clock rollback
  4. Hi-lo blocks: locality, allocator, gaps
  5. Client-minted id = idempotency key

basics

~20 s

Decide by where ids must be minted, whether ordering matters, and how much width you can afford. Options: random UUID (no coordination, poor locality), time-ordered UUID (locality plus decentralised minting), a timestamp-node-sequence 64-bit id (compact, needs node assignment and clock discipline), or centrally allocated ranges (best locality, needs an allocator).

solid answer

~50 s

I start from constraints rather than schemes. Must ids exist before any database contact? That rules out sequences. Do you need approximate creation ordering, or time-based partition pruning? That rules out pure randomness. Does the id fit in 64 bits, or can you afford 128 across every index and foreign key? Four credible answers. **UUIDv4**: zero coordination, worst index locality, no ordering. **UUIDv7 or ULID**: time prefix plus randomness, so decentralised minting with append-mostly inserts - my usual default when ids leave the database's control, accepting that creation time is visible. **Snowflake-style 64-bit** (timestamp, node id, per-millisecond counter): compact enough to keep bigint columns and roughly sortable, but you must assign node ids reliably and survive clock rollback. **Range allocation / hi-lo**: a central service hands each writer a block of sequential values, giving near-perfect locality and small keys, at the price of an allocator and gaps. Offline clients add one more requirement: the client-minted id doubles as the idempotency key for retried submissions.

code

text · 6 lines
text
| 1 bit  | 41 bits            | 10 bits  | 12 bits           |
| unused | ms since epoch     | node id  | seq within ms     |

41 bits of ms  -> ~69 years from the chosen epoch
10 bits node   -> 1024 generators
12 bits seq    -> 4096 ids per node per millisecond

go deeper

for a junior

Know that ids created outside the database must be globally unique without coordination, which is why UUIDs are used at all.

for a middle

Compare random versus time-ordered UUIDs on insert locality and width, and explain why a database sequence cannot serve offline clients.

for a senior

Evaluate Snowflake-style and range-allocation schemes concretely, including node-id assignment, clock rollback handling, gaps and the idempotency role of client-minted ids.

for a principal

Drive the decision from minting location, coordination tolerance, width budget, leakage and reversal cost; state explicitly that identifier choice is near-irreversible once ids propagate to foreign keys, exports and partners.

## Start with requirements, not schemes Six questions decide almost everything: 1. **Where is the id minted?** Database, application server, or an offline client? Anything minted outside the database rules out sequences and identity columns. 2. **Is coordination acceptable?** A central allocator is a dependency and a failure domain; some systems can afford it, region-partitioned or offline-first ones cannot. 3. **Does ordering matter?** For insert locality, for time-range pruning, or for the semantics of "most recent"? Note that no distributed scheme gives true global ordering; the best you get is approximate ordering under clock skew. 4. **What width can you afford?** 64 bits keeps `bigint` columns and every referencing index narrow. 128 bits doubles keys everywhere and, in clustered-primary-key engines, in every secondary index too. 5. **What may the id reveal?** Time prefixes leak creation time; node ids leak topology; counters leak throughput. 6. **What happens on retry?** If an offline client resends, the id must let the server recognise a duplicate. ## The options **Random UUIDv4.** Uniqueness with no coordination whatsoever, and no information leaked. Worst insert locality: random placement across the index means random reads, mid-page splits and index bloat once the index exceeds memory. Choose it when unpredictability of the value itself is the requirement. **Time-ordered UUIDv7 (or ULID/KSUID).** Millisecond timestamp in the high bits, randomness below. Still mintable anywhere with no coordination, but inserts cluster at the right edge, so index density and cache behaviour approach a sequence. Costs: 128 bits, visible creation time, and right-edge latch contention under heavy concurrent insert. Ordering across nodes is only as good as clock synchronisation - though uniqueness never depends on the clock, because the random bits carry it. **Snowflake-style 64-bit ids.** Partition the bits: timestamp since a custom epoch, a node identifier, and a per-millisecond sequence. Fits `bigint`, is roughly time-sortable, and needs no allocator per id. Costs are operational: node ids must be assigned uniquely and durably (config, coordination service, or a stable ordinal from the orchestrator), and you must decide what to do when the clock goes backwards - wait it out, or refuse to issue. Exhausting the per-millisecond counter means blocking until the next millisecond. The epoch and bit split also cap the scheme's lifetime and node count, which must be chosen with decades in mind. **Range allocation (hi-lo).** A central sequence or service hands each writer a block of, say, 1,000 values; the writer consumes them locally. Excellent locality per writer, small keys, no per-id coordination. Costs: an allocator to run and make highly available, gaps whenever a writer restarts with a partly used block, and ids that are only ordered within a writer. **Composite keys.** `(shard_id, local_sequence)` or `(tenant_id, local_sequence)` makes the routing information part of identity, which suits sharded systems and makes the shard obvious from the key. Costs: composite keys propagate into every foreign key, resharding is painful because the shard component is baked into identity, and per-tenant sequential numbers are visible to that tenant - occasionally desirable, occasionally a leak. **Per-node offset sequences.** Each node's sequence starts at a distinct offset and increments by the node count. Cheap, but adding nodes later is awkward and the scheme quietly assumes a fixed node count. ## Offline clients When a client creates rows without connectivity, the client-minted id is not merely an identifier - it is the deduplication key for retried submissions. Store it with a unique constraint so the second delivery of the same creation is rejected by the database rather than by hopeful application logic. This is a strong argument for UUIDv7 in mobile and field-service systems: the client can mint it, the server can insert it, and the unique constraint provides idempotency for free. ## Collision reasoning With 122 random bits, UUIDv4 collision probability is negligible at any realistic volume. UUIDv7's randomness is smaller because the timestamp consumes bits, so collisions are governed by how many ids a single generator can produce within one millisecond - real implementations add a monotonic counter within the millisecond precisely for this. Snowflake ids have no randomness at all: uniqueness rests entirely on node ids being unique and the clock never repeating a millisecond. That is a very different risk profile - a duplicated node id, from a bad deployment or a cloned VM, silently produces duplicate keys. ## How I would decide If ids can be minted by the database and there is one writer, use `bigint` identity and stop. Once minting must happen elsewhere, default to UUIDv7 stored natively, because it buys decentralised generation, acceptable insert locality and idempotency with no infrastructure. Move to Snowflake-style ids when 128-bit keys are measurably too wide for the data volume and you have the operational maturity to assign node ids and manage clocks. Use range allocation when a central allocator is already acceptable and you want small ordered keys. Reach for composite shard-scoped keys only when the shard is a genuinely permanent property of the row. And say the meta-point: this decision is very expensive to reverse, because ids propagate into every foreign key, every export, every partner integration and every URL. Decide it deliberately and early, then leave it alone.

  • What happens to a Snowflake-style generator when the machine clock jumps backwards, and how do you handle it?
    Timestamps it already issued would be re-used, so the scheme could emit duplicate ids for the same node. The standard handling is to remember the last timestamp issued and, if the clock is behind it, either block until the clock catches up when the drift is small, or refuse to issue ids and fail loudly when the drift is large. Silently continuing is the dangerous option, because the resulting duplicates are only discovered when a primary key insert fails or, worse, when data is merged later.
  • Why is a client-generated identifier especially valuable for offline-capable applications?
    The client can construct a complete object and link related records before it ever reaches the server, so the local data model is coherent without a round trip. When connectivity returns and a submission is retried, the same identifier arrives again, and a unique constraint on it makes the duplicate insert fail deterministically. That turns idempotency into a database guarantee instead of application-level guesswork about whether the earlier attempt succeeded.
  • What is the drawback of making the shard identifier part of the primary key?
    It bakes a placement decision into identity, so moving a row to another shard changes its key, which is exactly the mutable-key problem with all its propagation and stale-reference consequences. It also means every foreign key becomes composite, widening indexes and complicating queries. It is only right when the partitioning attribute, such as tenant, is genuinely permanent for the row's lifetime.

saying these in an interview costs you the question

  • Assuming a time-ordered scheme guarantees globally correct ordering across nodes despite clock skew
  • Deploying Snowflake-style generators without a durable, unique node-id assignment mechanism
  • Ignoring that 128-bit keys propagate into every foreign key and secondary index
  • Treating id-generation choice as easily reversible later
  • Overlooking that a client-minted id should serve as the idempotency key for retried writes

context