A versioning-enabled S3 bucket's storage bill keeps climbing even though the team has a lifecycle rule that expires objects after 30 days. Why is the data not going away, and what would you change?
answer
- the rule works, it just hides
- current-version actions vs noncurrent-version actions
- every expiration adds one more marker
- keep the last N as a safety net
- the markers need a rule of their own
basics
~20 sOn a versioned bucket, an Expiration rule only makes the current version noncurrent by adding a delete marker; the bytes stay as noncurrent versions and keep billing. Add NoncurrentVersionExpiration, plus a rule with ExpiredObjectDeleteMarker to sweep the markers.
solid answer
~50 sThe rule is working — it just does not mean what the team thinks. On a versioning-enabled bucket, `Expiration` with `Days: 30` applies to the *current* version: S3 inserts a delete marker so the key reads as 404, and the previous current version becomes noncurrent. Nothing is freed. Noncurrent versions live forever unless a separate `NoncurrentVersionExpiration` element removes them, and delete markers whose last remaining version is gone linger as expired object delete markers that clutter listings. The fix is three elements: `NoncurrentVersionExpiration` with `NoncurrentDays` (optionally `NewerNoncurrentVersions` to keep the last N as a safety net), `NoncurrentVersionTransitions` to move old versions to a cheaper class first, and a second rule with `Expiration: { ExpiredObjectDeleteMarker: true }` — it cannot share a rule with a `Days` expiration. Confirm the split first with S3 Storage Lens, which reports noncurrent-version bytes separately.
code
json · 23 lines{
"Rules": [
{
"ID": "expire-and-reap-versions",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Expiration": { "Days": 30 },
"NoncurrentVersionTransitions": [
{ "NoncurrentDays": 7, "StorageClass": "GLACIER_IR" }
],
"NoncurrentVersionExpiration": {
"NoncurrentDays": 30,
"NewerNoncurrentVersions": 2
}
},
{
"ID": "sweep-expired-delete-markers",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
]
}go deeper
Know that in a versioned bucket old versions keep costing money, and that a plain expiration rule does not remove them.
Explain the split between current-version and noncurrent-version lifecycle actions, and name NoncurrentVersionExpiration and ExpiredObjectDeleteMarker as the elements that actually reclaim space and tidy up.
Diagnose first with Storage Lens or ListObjectVersions, then roll the fix out per prefix with NewerNoncurrentVersions as a safety net, knowing that noncurrent expiration is an irreversible delete and that Object Lock overrides it.
Own the standard: a default retention policy for every versioned bucket in the estate, a recovery objective that justifies the number of versions kept, and cost allocation that makes noncurrent-version growth visible to the team creating it.
## Diagnose before you change anything The first move is to prove where the bytes are. A versioned bucket has three populations — current versions, noncurrent versions, and delete markers — and the console's object count shows only the first. Use `ListObjectVersions` on a sample prefix, or S3 Storage Lens, which reports **noncurrent version bytes** as its own metric alongside current-version bytes. If noncurrent bytes dominate, this diagnosis is confirmed and the rest follows. ```bash aws s3api list-object-versions --bucket my-bucket --prefix logs/2025/ \ --query 'length(Versions)' aws s3api list-object-versions --bucket my-bucket --prefix logs/2025/ \ --query 'length(DeleteMarkers)' ``` ## Why a normal expiration rule frees nothing Lifecycle actions on a versioned bucket are split into current-version and noncurrent-version actions, and they are not interchangeable: | Element | Acts on | Effect on a versioned bucket | |---|---|---| | `Expiration` with `Days` | the current version | inserts a delete marker; the old current version becomes noncurrent — **no bytes freed** | | `NoncurrentVersionExpiration` | noncurrent versions | permanently removes them — this is what frees storage | | `Transition` | the current version | changes its storage class | | `NoncurrentVersionTransitions` | noncurrent versions | changes their class, e.g. to an archival tier | | `Expiration` with `ExpiredObjectDeleteMarker: true` | delete markers with nothing behind them | removes the leftover markers | So a team that copies an unversioned bucket's lifecycle configuration into a versioned bucket gets a bucket where objects *appear* to expire while the bill grows monotonically. The `Days` rule is even actively contributing: every expiration creates one more delete marker. ## The corrected configuration Two rules, because S3 does not allow `Days` and `ExpiredObjectDeleteMarker` in the same `Expiration` element: ```json { "Rules": [ { "ID": "expire-and-reap-versions", "Status": "Enabled", "Filter": { "Prefix": "logs/" }, "Expiration": { "Days": 30 }, "NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 2 } }, { "ID": "sweep-expired-delete-markers", "Status": "Enabled", "Filter": { "Prefix": "logs/" }, "Expiration": { "ExpiredObjectDeleteMarker": true } } ] } ``` Three details worth knowing: - **`NoncurrentDays` is counted from the moment the version became noncurrent**, not from when it was uploaded. A version that stayed current for a year and was superseded yesterday has an age of one day for this rule. - **`NewerNoncurrentVersions` keeps the most recent N noncurrent versions regardless of age.** It converts "delete everything older than 30 days" into "delete everything older than 30 days, but always keep the last two", which is the safety net most teams actually want. - **An expired object delete marker** is a marker whose key has no noncurrent versions left behind it. Once `NoncurrentVersionExpiration` has swept the real versions, the markers become expired ones — which is why the sweep rule matters and why it does useful work only after the version expiration has run. ## Why the leftover markers matter beyond tidiness A prefix accumulating large numbers of delete markers makes `ListObjectVersions` slower and noisier, and paginated listings can return sparse pages while S3 scans past them — which surprises ETL jobs that assume a short page means the end of the data. Cleaning the markers keeps listing behaviour predictable. ## Rollout, because lifecycle deletes are irreversible Lifecycle expiration of a noncurrent version is a permanent delete — no marker, no undo. So: 1. Apply the rule to one prefix first with a generous `NoncurrentDays`, and watch Storage Lens before widening it. 2. Keep `NewerNoncurrentVersions` set so a bug that rewrites every object daily cannot flush the entire recovery window in one pass. 3. Remember that lifecycle runs asynchronously — objects are queued for expiration and removed afterwards, and billing stops when they are actually removed, so the bill responds over hours, not instantly. 4. If the bucket is a replication source or destination, decide deliberately whether the same retention applies on both sides; lifecycle configuration is per bucket and does not replicate. 5. If the bucket has Object Lock retention, locked versions are not expired by lifecycle until their retain-until date passes — the lock wins, and the rule silently reclaims less than you expect. ## The one-sentence answer "On a versioned bucket, `Expiration` hides; `NoncurrentVersionExpiration` deletes" — and if you only ever configured the first one, the bucket has been accumulating every version you ever wrote.
- How is NoncurrentDays counted — from upload, or from something else?From the moment the version stopped being current, not from its upload time. A version that stayed current for a year and was superseded yesterday counts as one day old for the rule. That is why a burst of overwrites resets the clock for a large number of versions at once and the reclamation lands later than people expect.
- Why set NewerNoncurrentVersions rather than relying on NoncurrentDays alone?Because a bug that rewrites every object repeatedly can push your entire recovery window past the age threshold in a single pass. NewerNoncurrentVersions always retains the most recent N noncurrent versions regardless of age, turning a pure time-based policy into one with a guaranteed floor of recoverable copies.
- The bucket also has Object Lock retention. What happens to the expiration rule?Locked versions are not removed by lifecycle until their retain-until date passes — the lock takes precedence and the rule silently reclaims less than expected. Plan retention and lifecycle together: use noncurrent-version transitions to a cheap archival class for the locked window, and let expiration take effect only once retention lapses.
saying these in an interview costs you the question
- Thinks Expiration Days deletes data on a versioned bucket
- Believes suspending versioning shrinks the existing bill
- Assumes delete markers are cleaned up automatically
- Puts Days and ExpiredObjectDeleteMarker in one Expiration element
- Expects the bill to drop the instant a lifecycle rule is applied