As the engineer accountable for a site's speed, how would you decide which third-party vendors are worth their performance cost, and what would you do about the ones that are not?
answer
- cost and value in one table
- named owner, stated purpose, review date
- holdback tests the claimed value
- tier decides the load strategy
- allowlist plus monitoring stops regrowth
basics
~20 sPut cost and value in the same table: measure each vendor's requests, bytes and main-thread time, then run a holdback to see whether its claimed business value survives its absence. Keep what pays, contain the rest by tier, and make admission a reviewed decision.
solid answer
~50 sI make it an explicit ledger rather than an argument about taste. Cost side: every vendor is measured the same way, in requests, transferred bytes, main-thread milliseconds and the change in field metrics when it is blocked. Value side: every vendor has a named business owner who states what decision it informs, and where the claim is contestable I run a holdback, withholding it from a slice of traffic and comparing the business metric that justifies it. A surprising number of vendors do not survive that test. Then I sort by tier: things the business genuinely cannot operate without load early; measurement moves server-side or gets sampled; engagement widgets get facades or load on intent; anything unowned or unproven is removed. I make admission deliberate, so a new vendor needs a review and an allowlist entry rather than a paste, and I keep field monitoring by origin so the list cannot quietly regrow.
go deeper
Know that third parties should be inventoried and that each one needs a reason to be there; be able to name the obvious containment options such as loading on demand.
Explain how you would price a vendor consistently, in requests, bytes and main-thread time, and which containment fits which kind of vendor.
Demonstrate the removal experiment: block or hold back the vendor, measure both performance and the business metric, and act on the result rather than on intuition.
Own the whole programme: comparable cost accounting, holdbacks as the arbiter of value, tiered load policy, technical admission control, continuous monitoring, and the cross-team bargain that keeps it alive.
## The failure this is meant to prevent Third-party scripts accumulate because each individual decision is cheap and locally reasonable, while the cost is shared, delayed and owned by nobody. The engineer who objects has a Lighthouse score; the stakeholder who wants the tag has a business need. Scores lose that argument, and should. The job at this level is to make the two sides commensurable. ## Cost in comparable units Every vendor gets measured identically, so the table can be sorted: number of requests and distinct origins, transferred bytes, main-thread time on a mid-tier device, and the delta in the field metrics that matter when the vendor's origins are blocked and the same journey repeated. Measure at the 75th percentile on hardware and networks resembling your users', because third-party execution cost scales with CPU and the difference between a developer laptop and a mid-range phone is where these arguments are usually won or lost. Cost is not a single number per vendor either. Separate the load-time cost from the ongoing cost: a widget with a modest download that installs document-level listeners charges you on every interaction for the life of the session, which shows up in interaction latency rather than in load metrics. ## Value that is claimed versus value that is demonstrated Every vendor should have a named business owner and a sentence stating which decision or capability depends on it. That alone retires a meaningful fraction of an old inventory: campaigns that ended, tools that were replaced, experiments whose owner has left. For the rest, the honest instrument is a holdback. Withhold the vendor from a slice of traffic, run it long enough to be statistically meaningful, and compare the business metric its owner cites. This is the move that changes the conversation, because it puts the burden of proof on the claim rather than on the objection, and because vendors often fail it: the chat widget that supposedly drives conversion, the personalisation tool whose lift disappears under scrutiny, the third analytics package measuring what two others already measure. A holdback also measures the other direction: run it and you learn what the page is worth without the vendor, in the same metrics you already report. ## Sorting by tier, not by argument Once cost and value are on the table, the decisions become mechanical: - **Essential to function** (payments, consent management, fraud, authentication): load early, keep them small, self-host or move first-party where the vendor permits, and hold them to the tightest availability requirements because they can break the page. - **Measurement**: prefer one implementation over four. Move to server-side collection where the vendor supports it so the device stops paying for each SDK, and sample where full fidelity is not needed. - **Engagement widgets** (chat, video, maps, social): facades and load-on-intent by default, because engagement rates are low and the cost per view is high. - **Unowned or unproven**: remove. Removal is a real option and should be exercised regularly enough that people expect it. ## Making the decision stick Governance decays unless it is enforced and monitored. Admission should be a reviewed change, with a Content Security Policy script-source allowlist making it technically true that an unapproved origin cannot run: the policy then doubles as an always-current inventory. Monitoring should watch third-party requests, bytes and long-frame attribution grouped by origin in real-user data, with an alert on a new origin, because vendors change their payloads without any deploy of yours and a periodic audit is stale the day after it is written. And there should be a standing review, quarterly or so, that re-asks the ownership question, since the inventory regrows to its old size within a year otherwise. ## The organisational half The technically correct programme fails if it is delivered as engineering vetoing marketing. What works is offering something in exchange: a fast, reliable path to add a vendor when it is justified, help with server-side collection so their data quality survives, and honest evidence when a holdback shows the vendor really does pay for itself, in which case you keep it and say so. Credibility earned by keeping a proven vendor is what makes the removal of an unproven one uncontroversial. The answer an interviewer is listening for is not zero third parties. It is that you can price them, prove their value, contain the ones you keep, and run a process that keeps the total from drifting.
- What is a holdback in this context, and why is it more persuasive than a performance profile?You withhold the vendor from a slice of real traffic and compare both the performance metrics and the business metric its owner cites. It answers the question people actually disagree about, which is whether the vendor earns its cost, rather than only how many milliseconds it consumes.
- How do you keep a cleaned-up third-party inventory from regrowing within a year?Make admission a reviewed decision backed by a technical control, such as a script-source allowlist that blocks unapproved origins, monitor origins continuously in real-user data with an alert on anything new, and hold a standing review that re-asks who owns each vendor and removes the unowned.
- A holdback shows a vendor genuinely improves conversion despite being expensive. What then?Keep it and say so publicly, then spend the effort on containment rather than removal: load it later or on intent, self-host or move it server-side if the vendor allows, and hold its budget against the vendors that failed the same test. Credibility from keeping a proven vendor makes the other removals easier.
- How would you present all this to executives without turning it into a scores conversation?One table: per vendor, its cost in requests, bytes and main-thread milliseconds, its measured effect on the metrics users feel, its named owner and the business result it demonstrated. Then a recommendation per row. The argument is about return on a shared budget, not about a tool's score.
saying these in an interview costs you the question
- Arguing from a Lighthouse score instead of business evidence
- Treating every third party as illegitimate by default
- Auditing once with no admission control afterwards
- Accepting a vendor's claimed value without testing it
- Measuring only on fast devices and office networks