A Kamal deploy has shipped a bad release. What does `kamal rollback` actually do, and in which situations will it not save you?

## What the command does `kamal rollback <version>` takes the version identifier of a release you previously deployed — the git revision that tagged the image — and makes that the running version again. Mechanically it is the ordinary cutover run backwards: on each host, bring up the container for that version, wait for its healthcheck, switch the proxy to it, then drain and stop the container that was serving. There is no build, no push, and usually no pull, because that container is typically still sitting stopped on the host from the earlier deploy. That is the reason rollback is measured in seconds while a fix-forward deploy is measured in minutes, and it is the single most valuable property of the tool during an incident. You need the version string. `kamal app containers` lists what each host actually has, and `kamal details` shows what is currently running — do not guess a SHA under pressure. ## Why it is fast: retained containers Kamal does not delete the previous container the moment it stops serving. It keeps a bounded number of recent ones per host — the `retain_containers` setting — precisely so that rollback is a local operation. That bound is also the first way rollback lets you down: **deploy often enough after a bad release and the good version ages out**. Ship five hotfixes trying to patch a broken release and the version you actually want to return to may no longer be on the host. It can still be pulled from the registry if the tag exists there, but that is a slower path and depends on your registry retention policy not having pruned it. ## The five ways it does not save you 1. **The container is gone.** Aged out by `retain_containers`, or pruned by a housekeeping job, or the image tag expired from the registry. 2. **The host never had it.** A server added to `servers:` after that version shipped has no such container. Mixed fleets after a partially failed deploy have the same shape of problem: some hosts can roll back locally, others cannot. 3. **A migration already ran forward.** This is the big one. Kamal rolls back the *application container*; it does nothing to your database. If the bad release renamed a column or dropped one, the old code now runs against a schema it does not understand, and you have converted a broken feature into a hard outage. The defence is entirely upstream: expand-and-contract migrations, never destructive in the same release as the code that stops using the column. 4. **Configuration moved.** Environment variables and secrets are managed separately from the image. If the bad release also changed an env var or rotated a secret, rolling the container back leaves the *new* configuration in place against the *old* code. 5. **Everything outside the app container.** Accessories — Postgres, Redis — are not versioned with the app and are not touched. Neither is the proxy's own configuration, nor anything you changed on the host by hand, nor any side effect already committed to a third-party system. ## How to talk about it in an interview The strong answer separates two things that candidates habitually merge: **the deploy is reversible, the release is not**. Kamal makes reversing the container trivially cheap, and that genuinely raises how confidently a small team can ship. It does nothing about the irreversible parts — schema changes, consumed messages, emails sent, money moved — and those are the parts that actually decide whether an incident is a two-minute blip or an afternoon. So the operational discipline that surrounds Kamal is the same one that surrounds any rolling release: make migrations backward compatible, keep configuration changes in their own release, and know the version string before you need it. ```bash kamal app containers # which versions each host still has kamal rollback 9f3c1a2 # health-gated cutover back to that version ``` One more practical note: rollback takes the same deploy lock as a deploy, so it will refuse to run while another deploy is in flight — which is a feature, not an obstacle, during an incident with several people at keyboards.