How would you measure the return on investment of a design system, and why is that measurement hard?
answer
- value minus cost, over cost
- time saved per reused part
- defects and cost of change
- what would have happened without it
- benefits lag the spending
basics
~20 sCompare the system's full cost with the value it creates: effort saved when teams reuse parts instead of building them, fewer UI and accessibility defects, and cheaper cross-cutting changes. It is hard because the counterfactual is unobservable and benefits lag costs.
solid answer
~40 sI would total the **cost** (the system team, tooling, and the migration effort consuming teams pay) and estimate the **value** in the same unit, usually engineering and design time: effort saved each time a team reuses a component rather than building it, fewer UI and accessibility defects reaching production, and the cost of cross-cutting changes such as a brand refresh with and without the system. ROI is then value minus cost, divided by cost, tracked over time. The measurement is hard because the **counterfactual** is unobservable, time savings are often self-reported, quality gains are difficult to price, and benefits **lag** spending. So I would use a few controlled comparisons, state assumptions openly, and present ranges rather than one precise number.
go deeper
Recall that ROI compares value with cost, and name time saved through reuse and fewer defects as the typical value measures.
Explain how to estimate saved effort per reused part and why migration cost belongs on the cost side even though consuming teams pay it.
Show how you would get observed data through controlled comparisons and defect trends, and present ranges with assumptions that survive a sceptical finance review.
Decide which value leadership should judge the system by, and set expectations about the lag between spending and return before anyone measures.
## What return on investment means here **Return on investment (ROI)** compares what an investment produced with what it cost: value minus cost, divided by cost. For a design system both sides must be expressed in the same unit, usually money or its proxy, engineering and design time. The question tests whether a candidate can name credible measures on both sides and be honest about their weaknesses. ## The cost side Costs are the easier half, but they are routinely under-counted: - **System team time**: designers, engineers, and anyone else working on the system. - **Tooling and infrastructure**: documentation site, build and release pipeline, testing. - **Migration effort** paid by consuming teams when they replace their own UI with system parts. - **Coordination time**: consuming teams waiting for or negotiating changes. ## The value side | Measure | How to estimate it | Weakness | |---|---|---| | Build effort saved | Time to build a part from scratch, times the number of teams that reuse it instead | Estimates of the from-scratch time are guesses | | Feature delivery time | Compare comparable features or screens built with and without the system | Few truly comparable pairs exist | | UI and accessibility defects | Defect counts per release in areas using system parts, before and after | Other changes happen at the same time | | Cost of cross-cutting change | Effort for a brand refresh or new platform, with system versus a past change without it | Rare events, small sample | | Design and review time | Time spent specifying or reviewing screens assembled from system parts | Often self-reported | Adoption itself, meaning how much of the product uses system parts, is a precondition for any of this value and is tracked separately; the value measures above only make sense once reuse is real. ## A worked estimate A streaming service's TV and web apps each need a media tile with focus, loading and error states. Suppose a team estimates a from-scratch build at several weeks, and five apps reuse the system's tile instead. The saved effort is roughly the build estimate times five, minus each app's integration time, minus the system team's share of maintenance for that tile. Summed across components and compared with total system cost, that gives a first ROI range. The value lies less in the arithmetic than in stating each assumption so leadership can challenge it. ## Why the measurement is hard 1. **The counterfactual is unobservable.** Nobody builds the same product twice, once with and once without the system. 2. **Attribution is muddy.** Faster delivery might come from the system, from a better team, or from simpler features. 3. **Self-reported time is unreliable.** Surveys of saved hours are biased toward the answer respondents expect. 4. **Quality is hard to price.** Fewer accessibility defects reduce risk and help users, but converting that to money involves assumptions. 5. **Benefits lag costs.** Spending starts immediately; savings arrive once parts are stable and reused. Early measurements look worse than the long-run picture. ## Making it credible anyway - Run a few **controlled comparisons**: build one screen with the system and time it against a recent comparable screen built without. - Prefer **observed data** (defect trackers, delivery dates) over surveys where possible, and use surveys as a secondary signal. - Present **ranges and assumptions**, not a single precise figure. - Track over **several quarters** so the lag is visible as a curve rather than a failed first measurement. - Include **non-financial value** that leadership cares about, such as brand consistency and reduced accessibility risk, described plainly rather than forced into money. ## Common traps - **Measuring output instead of outcome.** The number of components published or documentation pages written measures the system team's activity, not the value consumers gain. - **Leaving migration off the cost side** because consuming teams pay it from their own budgets. The organisation still pays it. - **Double counting.** If a saved build is credited to delivery speed and again to reduced defects, the total overstates the return. - **Ignoring maintenance of the alternative.** Without the system, each team would also maintain its own copies; that avoided upkeep is part of the value, and forgetting it understates the return. - **One-time snapshots.** A single measurement taken during the heavy early spending looks like failure; a series shows the trend.
- Why prefer ranges over a single ROI figure?Every input, from the from-scratch build estimate to the number of teams that would otherwise have built a part, is an assumption. A single precise figure implies a certainty that does not exist and collapses as soon as someone challenges one input. A range with stated assumptions invites leadership to adjust the inputs they doubt and still see whether the conclusion holds.
- How do you run a controlled comparison without building the product twice?Pick a screen or feature that is about to be built and has a recent comparable counterpart built without the system, then time the new build with system parts. Alternatively, have two engineers build the same small screen, one with and one without the system. Neither is a perfect experiment, but both produce observed data rather than recollection, which is more persuasive than surveys.
saying these in an interview costs you the question
- Counts only the system team's salary as the cost.
- Reports survey-based hours saved as a precise, audited figure.
- Judges the system's ROI on its first quarter alone.
- Treats component usage counts as the return itself.
- Attributes every delivery speed-up to the system without controls.