skip to content

You know every consumer of a rotated database password — how do you confirm each has moved before withdrawing the old value?

level: seniorimportance: must knowfreq 54%

answer

  1. silence is not proof
  2. start-up readers never read twice
  3. ask the accepting side
  4. which credential authenticated, not which was read
  5. force a re-read, then withdraw

basics

~20 s

Silence in the store's read records proves nothing: a process that read the value at start-up never reads again. Confirm from the accepting side — which credential each connection authenticated with — or force every holder to re-read.

solid answer

~50 s

There are two different records here and only one of them answers the question. The store's read record says who asked for which version and when; the accepting system's authentication record says which credential a connection actually presented. A consumer that read the old password at start-up and has held it ever since produces **no** further read record, so an absence of reads is ambiguous between "moved on", "still holding the copy it took in March" and "idle or dead". What ends the ambiguity is evidence of use, not evidence of reading: a period with no successful authentication using the outgoing value, covering the duty cycle of the slowest holder. Where the accepting system cannot distinguish which of two credentials was used — designs differ — the honest fallback is to force each known holder to re-read or restart, then withdraw inside a change window with a tested back-out.

code

json · 16 lines
json
{
  "storeReadRecord": {
    "valueName": "payments/db-password",
    "version": 7,
    "readBy": "payments-api",
    "readAt": "2026-03-04T06:11:20Z",
    "note": "last read by this consumer - it has held the value since"
  },
  "acceptingSideAuthRecord": {
    "identity": "payments_app",
    "credentialAccepted": "previous",
    "presentedFrom": "batch-runner-03",
    "at": "2026-09-19T02:00:44Z",
    "note": "a holder is still using the outgoing value - do not withdraw"
  }
}

go deeper

for a junior

The key fact to carry: a service usually reads its password once when it starts, so the absence of recent reads says nothing about whether it is still using the old one.

for a middle

Distinguish the two trails — reads at the store, authentications at the accepting system — and say which question each one answers.

for a senior

Lay out the evidence you would require before withdrawal, how long you would watch, and what you do when the accepting system cannot tell the two credentials apart.

for a principal

The lead's angle is what the estate must be able to prove before any withdrawal, and whether that capability is a requirement you place on systems you adopt.

## Two records, two different questions Rotation produces two trails, and candidates routinely reach for the wrong one. - **The store's read record** answers *who fetched this value, which version, and when*. It is a record of **reads**. - **The accepting system's authentication record** answers *which credential did this connection present*. It is a record of **use**. Withdrawal is safe when nothing is still **using** the outgoing value. Reads and uses are not the same event and are separated by the entire lifetime of a process. ## Why silence in the read record is ambiguous A long-running service reads its password once at start-up and holds it in memory for weeks. From the store's point of view that consumer goes quiet immediately after start-up and stays quiet — whether or not it is still hammering the database with the old value every second. So "no reads of the old version for two weeks" has at least three readings: 1. The consumer re-read, moved to the new value and is fine. 2. The consumer read once long ago, still holds the old value and is actively using it. 3. The consumer is idle, stopped or forgotten, and will surface the next time it runs. Only the first is safe to withdraw against, and the read record cannot tell you which one you are looking at. This is the specific trap in this material: an absence of evidence in the wrong trail is read as evidence of absence. ## What actually establishes disuse | Evidence | What it establishes | What it misses | |---|---|---| | No reads of the outgoing version | that nobody fetched it lately | every holder that read it once and kept it | | A read of the new version by each known consumer | that the value reached the process | whether the process reconnected with it | | No successful authentication with the outgoing credential, over a period covering the slowest holder | that nothing is using it | holders that are merely dormant, not moved | | Each known holder restarted or forced to re-read after publication | that no process can still be holding the old copy | copies taken outside the store by a person | The strongest single signal is the third: use, observed at the accepting side, absent for long enough. The strongest combination is the third plus the fourth — you removed the possibility, then watched to confirm. ## When the accepting side cannot tell you which credential was used This is common and the answer must not pretend otherwise. Some accepting systems record which of several credentials an identity authenticated with; others record only the identity, so both passwords look identical in the trail. Designs genuinely differ, and you plan against the one you have. Where the distinction is unavailable: 1. Force the issue instead of observing it — restart or re-trigger every known holder after the new value is published, so no process can still be holding the old copy. 2. Withdraw inside a change window with an owner watching, not on a Friday and not unattended. 3. Have a tested back-out: re-adding the old value at the accepting side, with the command already written down. 4. Watch authentication failures for that identity for a period that covers the slowest scheduled consumer, and treat a failure as the signal that a holder was missed — which it is. That is a weaker position than proof, and saying so is part of a good answer. ## Deleting is not withdrawing One more confusion worth naming. Removing the value from the store does not withdraw it. The store is where the value is **obtained**; the database is where it is **accepted**. Delete it from the store and every copy already delivered keeps authenticating exactly as before, while you have lost the ability to check the value or hand it back during a back-out. Withdrawal is an action against the system that accepts the credential, and it is the only action that ends the old value's life. ## What a good answer sounds like "I have the consumer list. I check that each one shows a read of the new version after publication — necessary, not sufficient. Then I watch the database's own record for authentications using the outgoing password, across a period that covers the monthly job, not just the request path. If the database cannot distinguish the two, I restart or re-trigger every holder instead, then withdraw in a change window with the back-out written down and someone watching the authentication failures for that account."

  • A known consumer shows no read of the new version at all. What are the possibilities?
    It is idle, stopped or decommissioned; it reads only at start-up and has not restarted since publication; or it holds a copy taken outside the store entirely. All three mean the same thing for the withdrawal: you do not yet know, and withdrawing will tell you the expensive way.
  • The database records the account but not which of its two passwords matched. Now what?
    You cannot observe disuse, so you remove the possibility instead: force every known holder to re-read or restart after publication, withdraw inside a change window with a written back-out, and watch authentication failures for that account through the slowest consumer's next run.
  • Every consumer has read the new version. Is that enough to withdraw?
    Not by itself. A read means the value reached the process, not that the process reopened its connections with it — some hold the original until a restart. Pair it with the absence of use at the accepting side, or with a forced restart of each holder.

A library's borrowing log tells you which copies went out and when; only the return desk tells you which are back. The store's read record is the borrowing log, and the accepting system's record of what authenticated is the return desk.

saying these in an interview costs you the question

  • Treats no reads of the outgoing version as proof nobody holds it.
  • Assumes a running process re-reads the store on every request.
  • Withdraws as soon as each consumer has read the new version once.
  • Believes removing the value from the store withdraws it.
  • Checks only the store's trail and never the accepting system's.
  • Withdraws unattended, with no back-out written down.