When does merging several build steps into one earn the coarser cache invalidation that comes with it?

## A step is two things at once Each step in a build is a **unit of reuse** - the granularity at which the builder decides to skip work - and a **layer** in the finished image. Deciding how many steps to write is therefore not a formatting question. It decides what an edit costs and what the image is made of. ## What splitting buys Splitting separates inputs. If two pieces of work have different volatility, putting them in separate steps lets the stable one keep hitting while the volatile one re-runs: - a dependency install whose inputs move weekly, separated from a compile whose inputs move hourly; - a slow one-off preparation step, separated from anything that changes per commit; - a step that copies a rarely-edited configuration file, separated from the step that copies the source. The gain is real only when the volatility genuinely differs. Splitting two pieces of work that always change together produces two cache entries that always miss together - more layers, the same rebuild. ## What merging buys, and costs Merging gives up granularity: **any** input to the merged step invalidates all of it. That is a genuine cost, and it is the reason a fetch and a compile should rarely share a step - a source edit would then re-run the fetch as well. It earns its place in three situations: 1. **The pieces share inputs anyway.** Two commands driven by the same file will always miss and hit together, so separating them only adds layers. 2. **An intermediate result must not persist.** Whatever a step leaves behind is in that step's layer, so work that produces scratch output and then cleans it up belongs inside one step; a clean-up in a later step cannot remove what an earlier layer already recorded. 3. **The pieces must stay consistent.** Where a half-applied result would be wrong - a fetch and the verification of what it fetched, a change and its fix-up - a single unit of reuse means you never assemble an image from one half of a stale pair and one half of a fresh one. | axis | more, smaller steps | fewer, merged steps | |---|---|---| | reuse after an edit | finer - stable work keeps hitting | coarser - the whole step re-runs | | layer count | higher | lower | | intermediate output | persists in its own layer | contained inside one layer | | consistency between pieces | can diverge across runs | always produced together | ## The decision, as a rule Split along **volatility boundaries**. Ask of any two adjacent pieces of work: *do their inputs change at different rates?* If yes, split - the stable one is worth protecting. If no, merging costs nothing in reuse and may buy containment or consistency. A useful secondary check is the direction of the cascade. Because a miss invalidates everything after it, a fine split that sits **after** the volatile step is decoration - those steps all re-run anyway. Splitting is only ever worth doing **before** the volatile work, where the reuse is still live. ## The failure modes at both extremes - **Over-split:** a long chain of trivial steps whose inputs all move together. Every build misses at the same point and runs the whole tail; the only result is more layers to stack. - **Over-merged:** one enormous step that does everything. Any edit anywhere re-runs all of it, which is the forty-seconds-to-nine-minutes shape with a different cause. Both are reachable from a well-intentioned rule applied without looking at the inputs, which is why the answer an interviewer wants is *the volatility of each piece's inputs*, and not a target number of steps.