What pinning policy do you set after a transitive dependency floats to a new release and breaks a video-metadata extractor's 27-minute suite with floating-point rounding drift?
answer
- The goal is choosing when, not freezing
- Intent, resolution and installation are separate layers
- Pipelines must never resolve afresh
- Libraries and applications get opposite rules
- Reproducible installs are not reproducible numerics
basics
~20 sSeparate intent from resolution: keep ranges in project metadata, commit a resolved lock for every deployable, install from the lock rather than re-resolving, and re-resolve on a schedule so upgrades arrive as a reviewed change.
solid answer
~50 sThe incident is not an argument for pinning everything forever; it is an argument for **deciding when a new version enters**. I would keep abstract ranges in each project's metadata as the statement of intent, commit a fully resolved lock beside every deployable, and make CI install from the lock so no pipeline run ever resolves afresh. Upgrades then arrive through a scheduled re-resolve that opens a reviewable change, with the 27-minute suite as the gate -- the drift is caught by a bot's change rather than by a release. For libraries the policy inverts: publish floors, add ceilings only where a release has actually broken you, and run one unpinned job against latest dependencies so you learn about breakage before consumers do. Two caveats I would state out loud: reproducible installs are not reproducible numerics, and a tool-owned lock format is a portability cost worth naming.
go deeper
Take away the shape of the answer: exact versions are recorded for what gets deployed, looser ranges are declared for what a library publishes, and upgrades should be a deliberate change rather than something that happens on its own.
Be ready to describe the three layers -- declared ranges, a generated lock, an install that only consumes the lock -- and to say why a pipeline that resolves afresh on every run produces results that depend on the date.
Demonstrate you have run this: the upgrade cadence, how a failing scheduled bump is triaged, how a hashed export covers tooling that cannot read a lock, and how you keep a slow suite as a real gate instead of an excuse to skip it.
Own the tradeoffs across the fleet: cadence versus review load, tool-owned locks versus a portable format, central constraints versus per-service autonomy, and the honest limit that pinning bounds when a change lands but never guarantees identical numeric behaviour.
### Read the incident correctly A transitive dependency moved, the numeric output shifted slightly, and a long suite failed. Nothing here is unusual; the only real defect is that an unreviewed version change reached a place where it could fail. A pinning policy is not a mechanism for freezing software -- it is a mechanism for **choosing the moment** at which a change lands, and for making sure a human sees a diff when it does. ### The layers, and who owns each **Intent** lives in project metadata: the distributions the project imports, with ranges, plus development-only sets in a dependency group so test tooling does not leak into what the service deploys. This is the file people edit and reviewers read. **Resolution** lives in a committed lock: the full closure with markers, per-artefact hashes and provenance. It is generated, never hand-edited, and its diff is the artefact reviewers actually study during an upgrade. **Installation** consumes the lock and nothing else. The single most valuable rule in the whole policy is that CI and deployment never re-resolve. A pipeline that resolves on each run is a pipeline whose result depends on the day it ran, which is precisely how a drift like this one arrives unannounced. Where the deployment tooling cannot read a lock, export it to a fully pinned, hashed requirements file at the boundary and install that under hash checking. The guarantee survives; only the file format changes. ### Cadence, not freezing A lock that is never regenerated rots: security fixes do not land, and the eventual upgrade is a hundred-package jump that nobody can review. So the policy needs a scheduled re-resolve -- weekly for most services -- that opens a change containing only the lock diff, gated by the full suite. When that job fails, it fails on a bot's change with a clear culprit list, and the 27-minute cost is paid by a machine on a schedule rather than by an engineer during a release. Grouping patch-level bumps into a single change and isolating majors keeps the review load sane. ### Libraries need the opposite discipline A published library's ranges become everyone else's constraints, so pinning there exports your problems. Declare the floor you genuinely need; add a ceiling only when a specific release has broken you, and remove it when fixed. Keep a lock in the repository for the library's own CI so its runs are deterministic, and add one job that deliberately installs the newest permitted dependencies -- unpinned. That job is the counterweight to pinning: it converts 'our users will hit this in three weeks' into 'our CI hit this this morning'. ### The portfolio dimension Across many repositories, three questions matter more than the file format. Who owns the upgrade queue when a floor must move everywhere -- a centrally published constraints file bounds an indirect package across services without inventing direct dependencies. Which index do artefacts come from -- an internal mirror plus hash-verified installs closes the substitution shape. And how much tool lock-in is acceptable -- a tool-owned lock is fine while everyone uses that tool, and the standard interchange lock format exists precisely for when they do not. ### The caveat worth stating out loud Pinning bounds this failure; it does not eliminate the class. Floating-point results can shift with a different compiled backend, different threading, different hardware, or a build of a source distribution that picked up different flags -- none of which a version pin controls. So alongside the policy I would ask whether a suite that fails on a last-place rounding difference is asserting the right thing. Exact-equality assertions on floating-point output are a fragility of their own; tolerance-based comparisons with an explicit, documented budget make the suite report real regressions instead of arithmetic noise. The pinning policy buys you the *when*; the assertions decide what counts as broken. ### What I would not do I would not pin the library's published metadata to make the incident go away, I would not disable the slow suite to reduce upgrade friction, and I would not leave a hand-added constraint in place with no expiry and no owner. Each of those trades a visible cost today for an invisible one later.
- Why is a lock that is never regenerated as dangerous as no lock at all?Because the risk moves rather than disappearing. Security fixes stop arriving, and the eventual upgrade becomes one enormous multi-package jump whose failures nobody can attribute. A lock is only safe in combination with a cadence: a scheduled re-resolve that keeps each diff small enough to review and each failure small enough to blame on a specific dependency.
- Your library must pin nothing, but its CI must be deterministic. How do you get both?Publish ranges in the metadata and keep a lock in the repository that only the library's own test jobs install from -- it never ships in the wheel, so consumers are unaffected. Then add a job that resolves the newest permitted versions with no lock at all. Determinism for the default path, early warning from the unpinned path.
- When would you accept the portability cost of a tool-owned lock format?When one toolchain is genuinely standard across the repositories that matter, and the lock is consumed by pipelines you control rather than by external parties. The cost bites when a team must switch tools or when something outside your fleet needs the resolution; exporting to a pinned, hashed requirements file, or to the standard interchange lock format, keeps that escape hatch open cheaply.
- How do you keep the scheduled upgrade change from being rubber-stamped?Make the gate meaningful and the diff small. Group patch bumps but isolate majors, require the full suite rather than a fast subset, and treat the lock diff as the review artefact so a reviewer sees which packages moved and why. If the change is merged unread every week, the pipeline is providing the illusion of review, not review.
saying these in an interview costs you the question
- Answers pin everything forever with no upgrade cadence
- Applies the application's pinning rules to a published library
- Lets CI re-resolve dependencies on every run
- Treats a lock as a security control on its own
- Assumes pinned installs guarantee identical numeric output
- Adds ad-hoc constraints with no owner or expiry