How would you build a defensible technical and business fitness scoring model for application portfolio management, and what makes such scores easy to game or misapply?

## What the model is A fitness scoring model is the quantitative engine behind frameworks like TIME: it converts qualitative judgments about an application into two comparable numbers — **business fitness** and **technical fitness** — so that hundreds of dissimilar applications can be ranked and compared on the same footing. Building one starts with defining a criteria list per axis. ## The criteria on each axis **Business fitness** typically includes: - **functional coverage** (does the app still do what the business needs, or has the business moved past its capabilities); - **user or customer satisfaction** (survey or NPS-style data); - **strategic alignment** (does it support where the company is heading, not just where it's been); - **criticality** (revenue impact, regulatory dependency, process centrality). **Technical fitness** typically includes: - **platform and vendor support status** (is it on a version still receiving patches, is the vendor still in business); - **security posture** (open CVE count, time-to-patch history); - **scalability headroom** (how close is it to a known ceiling); - **code quality and accumulated technical debt**; - **integration complexity** (how many other systems depend on it and how brittle those integrations are); - **total cost of ownership** relative to comparable systems. Each criterion is assigned a weight reflecting organizational priorities — a heavily regulated industry might weight security posture more than a startup would — and the weighted criteria combine into a single composite score per axis, usually on a 1-to-5 or 1-to-10 scale, which is what gets plotted for the TIME classification. ## Why a model rather than ad hoc judgment The reason such a model exists rather than relying on ad hoc judgment is straightforward: a portfolio committee cannot deeply understand the internals of every one of several hundred applications well enough to compare a payroll system against an internal reporting tool by intuition alone. A structured, repeatable scoring model gives every application the same evaluation lens, produces a number that survives a change in reviewer, and creates an audit trail that justifies funding or decommissioning decisions when challenged — which matters enormously when the decision affects someone's budget or job. ## The trade-offs The central design trade-off is quantitative versus qualitative criteria. | Criterion type | Strength | Weakness | |---|---|---| | **Quantitative signals** — CVE count, uptime percentage, license cost | Objective and hard to dispute | Don't capture strategic importance or nuance; a system with zero open vulnerabilities can still be strategically obsolete | | **Qualitative signals** — strategic alignment, business owner satisfaction | Capture that nuance | Inherently subjective and vulnerable to political pressure | A second trade-off sits in the weighting itself: weights encode organizational values, and those choices are themselves debatable. Weighting total cost of ownership heavily will systematically push expensive-but-valuable applications toward elimination even when they're strategically important, while weighting strategic alignment heavily can let genuinely broken, high-risk systems survive because someone influential vouches for their importance. ## Failure modes The most consequential failure mode is **self-scoring bias**: when the people who own an application are also the primary or sole source of its fitness scores, they have a direct incentive to inflate business fitness to protect budget and headcount, and to understate technical debt to avoid triggering a mandated migration project that consumes their team's time without a visible feature to show for it. 1. **A related failure is score-inflation creep** — over successive annual cycles, scores drift upward across the board as reviewers become reluctant to give anyone a low mark, until nearly everything clusters around 'average' and the scores lose their ability to discriminate between genuinely healthy and genuinely at-risk applications. 2. **A third failure is treating scoring as a compliance checkbox** — the exercise gets completed once a year to satisfy an audit requirement, the resulting numbers sit in a spreadsheet, and no funding or roadmap decision is actually tied to them, which teaches everyone in the organization that the exercise doesn't matter and further erodes score quality over subsequent cycles. 3. **A fourth failure is failing to refresh scores after a real change** — an application that underwent a technical rewrite and genuinely improved its technical fitness can sit at its old, stale low score in the portfolio tool for years because nobody re-ran the assessment, making the tool actively misleading rather than just outdated. ## A concrete scenario A concrete scenario: an enterprise architecture team scores an internal claims-processing application. The business unit that owns it self-reports high satisfaction and strategic importance to keep it out of the Migrate bucket, even though the application runs on unsupported Java 8 with three unpatched, publicly known vulnerabilities. An independent architecture review pulls the CVE data directly from the organization's vulnerability scanner rather than accepting the self-reported technical score, overrides the inflated number, and the resulting objective technical-fitness score is low enough to trigger mandated migration funding despite the business unit's resistance — illustrating why defensible scoring requires an evidence source independent of the people whose incentives run counter to an honest score.