Tiering four hundred million small log objects into a colder class made the storage bill rise — why?

seniorshould knowfreq 44%

answer

two dimensions, bytes and objects
the discount is per byte only
a floor billed for each object
tiny payload, tens of kilobytes billed
compact before you transition

basics

~20 s

Colder tiers bill a fixed overhead per object on top of its real size, and charge a fee per object moved. Across hundreds of millions of tiny objects that overhead dwarfs the payload, so the cheaper per-gigabyte rate applies to far more billed gigabytes.

solid answer

~40 s

Cold tiers are priced for few large objects. Two per-object charges break the assumption behind the rent discount. First, a colder tier adds a fixed per-object overhead — metadata billed as if each object were larger than it is; when the payload is a few kilobytes and the overhead is tens, the billed volume is many times the stored volume. Second, the transition itself costs a fee per object, and four hundred million of them is a large one-off charge that then has to be earned back. Multiply both by the object count and the discount can be smaller than the inflation. The fix is not a different tier but a different object layout: compact the small objects into a few large compressed archives, then transition those.

Cold tiers are priced for few large objects. Two per-object charges break the assumption behind the rent discount. First, a colder tier adds a fixed **per-object overhead** — metadata billed as if each object were larger than it is; when the payload is a few kilobytes and the overhead is tens, the billed volume is many times the stored volume. Second, the transition itself costs a **fee per object**, and four hundred million of them is a large one-off charge that then has to be earned back. Multiply both by the object count and the discount can be smaller than the inflation. The fix is not a different tier but a different object layout: compact the small objects into a few large compressed archives, then transition those.

code

pseudocode · 16 lines

pseudocode

objectCount        = 400_000_000
averageObjectBytes = 4 * KiB
perObjectOverhead  = 40 * KiB        // invented; each provider publishes its own

billedBytesWarm = objectCount * averageObjectBytes
billedBytesCold = objectCount * (averageObjectBytes + perObjectOverhead)

monthlyRentWarm = billedBytesWarm * warmRatePerBytePerMonth
monthlyRentCold = billedBytesCold * coldRatePerBytePerMonth
oneOffTransition = objectCount * transitionFeePerObject

if monthlyRentCold >= monthlyRentWarm then
    report "overhead inflation exceeds the tier discount; do not transition"
else
    monthsToBreakEven = oneOffTransition / (monthlyRentWarm - monthlyRentCold)
    report monthsToBreakEven, "compare against the retention window"

go deeper

for a junior

Recall that storage bills have a per-object side as well as a per-gigabyte side, and that very small objects are billed as if they were bigger in colder tiers.

for a middle

Explain the inflation factor: payload plus a fixed overhead, times the object count, and why the tier discount has to beat that ratio before it saves anything.

for a senior

Show the estimate you would run before applying the rule, and name compaction — not a colder tier — as the fix, including how you would pilot it on one prefix.

for a principal

Treat it upstream: the object count is set by how the writer emits data. Make buffered, compacted archival output the platform default so no team meets this per-object arithmetic on their own.

Why object count, not volume, decides this

Every charge in a storage tier falls into one of two families: per byte and per object. The rent discount you went looking for is per byte. Almost everything that bites in a colder tier is per object:

a fixed per-object overhead — colder tiers keep extra metadata per object and bill each object as though it carried some additional kilobytes;
a per-object transition fee — charged once when the rule moves the object;
a minimum storage duration applied per object;
per-object restore requests later, if anything ever reads it back.

A log archive is the pathological shape for all four: enormous object counts, tiny objects. The mental model that fails is "we have N terabytes and the cold rate is a fraction of the warm rate". The store does not bill you for N terabytes once the objects are cold; it bills you for N terabytes plus the overhead times the object count.

The arithmetic

Take an invented but plausible shape: 400,000,000 objects averaging 4 KiB, and a per-object overhead of 40 KiB in the colder tier. Neither number is any provider's; both are there to show the ratio.

Quantity	Warm tier	Colder tier
Bytes stored	~1.5 TiB	~1.5 TiB
Bytes billed	~1.5 TiB	~16.4 TiB
Inflation factor	1x	~11x

The colder tier now has to be more than eleven times cheaper per byte merely to break even on rent — before the transition fee, before the minimum storage duration, before any retrieval. Tier discounts are large but they are not usually that large, so the bill goes up. This is not an anomaly; it is the designed behaviour of a tier priced on the assumption that archives are made of few big files.

The fee, separately

Even where the overhead is tolerable, the one-off fee is charged per object moved. Four hundred million objects is four hundred million billable operations, incurred in a single month, to buy a monthly saving that may be a fraction of it. The break-even is:

monthsToBreakEven = oneOffTransitionFee / (monthlyRentWarm - monthlyRentCold)

If that denominator is small — because the overhead ate the discount — the break-even can exceed the data's retention window entirely. You then pay a large fee to save nothing, and delete the objects before the saving arrives.

What to do instead

Compact before you tier. A batch job that rolls a day of small log objects into one large compressed archive object turns 400 million objects into a few thousand. The overhead becomes negligible, the transition fee becomes trivial, and the rent discount finally applies to something close to the real volume.
Compress at the same time. Log text compresses heavily, so the compaction usually cuts the stored volume as well as the object count. Both effects push the same way.
Choose the compacted object size deliberately. Large enough that the per-object overhead is noise; small enough that a future read does not have to pull a month to get an hour. The granularity of a likely read is the right guide.
Estimate before the rule runs. Count the objects the rule's filter matches and multiply by the fee. Run the rule against one path prefix first and read the next invoice before applying it to everything.
Revisit the writer. The real defect is often upstream: a service writing one object per request or per minute. Buffering at the writer avoids the problem instead of cleaning up after it.

The judgment this question is testing

The interviewer is not looking for "cold tiers have overhead". They are looking for whether you reason about storage cost in two dimensions — bytes and objects — and whether you check the second before writing a rule that touches hundreds of millions of things. The give-away of a weak answer is proposing a colder tier as the fix, which multiplies every per-object charge again. The shape of the data has to change before the tier does.

## Why object count, not volume, decides this Every charge in a storage tier falls into one of two families: **per byte** and **per object**. The rent discount you went looking for is per byte. Almost everything that bites in a colder tier is per object: - a **fixed per-object overhead** — colder tiers keep extra metadata per object and bill each object as though it carried some additional kilobytes; - a **per-object transition fee** — charged once when the rule moves the object; - a **minimum storage duration** applied per object; - **per-object restore requests** later, if anything ever reads it back. A log archive is the pathological shape for all four: enormous object counts, tiny objects. The mental model that fails is "we have N terabytes and the cold rate is a fraction of the warm rate". The store does not bill you for N terabytes once the objects are cold; it bills you for N terabytes plus the overhead times the object count. ## The arithmetic Take an invented but plausible shape: 400,000,000 objects averaging 4 KiB, and a per-object overhead of 40 KiB in the colder tier. Neither number is any provider's; both are there to show the ratio. | Quantity | Warm tier | Colder tier | |---|---|---| | Bytes stored | ~1.5 TiB | ~1.5 TiB | | Bytes billed | ~1.5 TiB | ~16.4 TiB | | Inflation factor | 1x | ~11x | The colder tier now has to be more than eleven times cheaper per byte merely to break even on rent — before the transition fee, before the minimum storage duration, before any retrieval. Tier discounts are large but they are not usually that large, so the bill goes up. This is not an anomaly; it is the designed behaviour of a tier priced on the assumption that archives are made of few big files. ## The fee, separately Even where the overhead is tolerable, the one-off fee is charged per object moved. Four hundred million objects is four hundred million billable operations, incurred in a single month, to buy a monthly saving that may be a fraction of it. The break-even is: `monthsToBreakEven = oneOffTransitionFee / (monthlyRentWarm - monthlyRentCold)` If that denominator is small — because the overhead ate the discount — the break-even can exceed the data's retention window entirely. You then pay a large fee to save nothing, and delete the objects before the saving arrives. ## What to do instead 1. **Compact before you tier.** A batch job that rolls a day of small log objects into one large compressed archive object turns 400 million objects into a few thousand. The overhead becomes negligible, the transition fee becomes trivial, and the rent discount finally applies to something close to the real volume. 2. **Compress at the same time.** Log text compresses heavily, so the compaction usually cuts the stored volume as well as the object count. Both effects push the same way. 3. **Choose the compacted object size deliberately.** Large enough that the per-object overhead is noise; small enough that a future read does not have to pull a month to get an hour. The granularity of a likely read is the right guide. 4. **Estimate before the rule runs.** Count the objects the rule's filter matches and multiply by the fee. Run the rule against one path prefix first and read the next invoice before applying it to everything. 5. **Revisit the writer.** The real defect is often upstream: a service writing one object per request or per minute. Buffering at the writer avoids the problem instead of cleaning up after it. ## The judgment this question is testing The interviewer is not looking for "cold tiers have overhead". They are looking for whether you reason about storage cost in two dimensions — bytes and objects — and whether you check the second before writing a rule that touches hundreds of millions of things. The give-away of a weak answer is proposing a *colder* tier as the fix, which multiplies every per-object charge again. The shape of the data has to change before the tier does.

How would you size the compacted archive objects?
By the granularity of a plausible future read. Large enough that the per-object overhead and fees are noise — typically hundreds of megabytes rather than kilobytes — but not so large that answering a one-hour question means retrieving and paying for a month. One object per source per day or per hour is a common landing point.
How do you estimate the transition fee before running the rule?
Count the objects the rule's filter would match, not the bytes, and multiply by the per-object fee. Object counts are usually available from the store's own metrics or an inventory listing. Then apply the rule to one path prefix first and compare the next invoice against the estimate.
Does the same overhead apply in the warm tier?
The warm tier generally bills close to the real object size, which is why the problem only appears after the transition. The per-object overhead is a property of the colder tiers, which are designed around large archival objects — so the same data can be economical warm and uneconomical cold.

saying these in an interview costs you the question

Reasons about storage cost only in gigabytes, never in object count
Proposes an even colder tier when the bill rises
Assumes billed size equals stored size in every tier
Forgets the transition fee is charged per object, not per gigabyte
Believes compaction only helps compression, not billing

Tiering four hundred million small log objects into a colder class made the storage bill rise — why?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Why object count, not volume, decides this

The arithmetic

The fee, separately

What to do instead

The judgment this question is testing

▾common follow-ups3 questions

▾red flags5 flags

Why object count, not volume, decides this

The arithmetic

The fee, separately

What to do instead

The judgment this question is testing

answer

go deeper