At 14:32 someone ran an UPDATE without a WHERE clause against the production database. Walk through recovering the database to the state just before that statement.

## Minute one: contain, do not repair The first mistakes are usually made in the first five minutes. Stop application writes - drop the database out of the load balancer, revoke write access, or put the service in maintenance mode. Every further write both increases the data you must reconcile and, if you later cut over to a restored copy, increases what you throw away. Equally important: **do not touch the damaged instance destructively**. Do not restore over it, do not reinitialise it, do not let anyone 'fix it with an UPDATE'. It contains the most recent log records, it is your only copy of the valid work done after 14:32, and it is the fallback if the restore turns out to be unusable. Then force the current log segment to be archived and confirm the archive is contiguous up to the present. If a log segment covering the window is missing, your options collapse immediately and you need to know that before you spend an hour restoring. ## Finding the exact target 'Just before 14:32' is not precise enough. Sources for a precise target: - Application, proxy or audit logs recording the statement and its transaction identifier or commit time. - Reading the archived log itself with the engine's log-inspection tooling to find the offending transaction and the log position immediately preceding its commit. - A named restore point, if someone had the discipline to create one before a risky operation. A **log position** (log sequence number, or file plus offset) is exact and monotonic. A **timestamp** depends on the clock at commit time and typically has second-level practical resolution, so a timestamp target can easily include or exclude neighbouring transactions you did not mean to touch. Use the timestamp only to narrow the search, then convert to a position. Note what recovery can and cannot do: it stops at a transaction boundary, so 'just before the statement' means 'after the last transaction that committed before it'. Any other transaction that committed in the same instant is on one side of the line or the other; you do not get to pick per-transaction. ## Performing the restore Provision a **separate** target: a spare host, a fresh volume, a cloud instance. You need enough space for the base backup plus the fetched log segments plus room to work. Restore the most recent base backup whose start time is **before** the target. Make the archive reachable so recovery can fetch segments on demand. Configure: - the recovery **target** (the position or timestamp you determined), - whether the target itself is **included or excluded** - you want to stop *before* the offending transaction, - the **action at target**: pause rather than promote, so you can look before committing to the outcome. Start recovery. Replay first reaches the consistency point (the end of the base backup) and only then becomes openable; after that it continues to your target. Expect the wall-clock duration to be dominated by copying the base backup plus replaying however many hours of log have accumulated since it was taken. ## Verify before promoting With recovery paused at the target, open the database read-only and check: - the rows the bad statement damaged now hold their pre-damage values, - a row you know was written *shortly before* 14:32 is present - proof you did not stop too early, - a row you know was written after 14:32 is absent - proof you did not stop too late, - basic integrity on a couple of unrelated tables. If you overshot or undershot, you re-run recovery with an adjusted target. Depending on the engine this may mean restoring the base backup again, which is why you never destroyed the original and why you sized the restore host with room to repeat. ## The cut-over decision Two outcomes, and the choice is a business decision, not a DBA one: - **Promote the restored copy** as the new production. Simple and fast to reason about, but every valid transaction committed between 14:32 and now is lost. Once promoted, the instance starts a new branch of history; treat old replicas as invalid and take a fresh base backup immediately. - **Keep production and repair surgically.** Export just the affected rows from the read-only restored copy and merge them into the live database, reconciling rows that were legitimately changed after 14:32 and being careful with keys, sequences and foreign-key order. This preserves the intervening work but requires care and is only feasible when the blast radius is well bounded. Whichever you choose, state the resulting data loss plainly, and afterwards close the loop on the cause: why an unqualified UPDATE could be run against production at all, whether safe-update guards, mandatory review, restricted write accounts or pre-change restore markers should exist.