skip to content

Target Selection

Deciding what a suite runs on and what it runs there: the mix of platform versions, capability tiers and screen classes to cover, and which build of the app is under test on them.

on this pageshow

questions

7

Why can't a handheld app's automated suite just target the newest platform version?

level: middleimportance: must knowfreq 58%

answer

  1. Nobody can force an update
  2. Old devices stop receiving platform updates
  3. The tail grows as the product ages
  4. Newest, most-used, oldest supported

basics

~20 s

Handheld users choose when to update, and many devices stop receiving platform updates entirely, so the installed base stays spread across several version lines for years. A suite that runs only on the newest line leaves most real users unverified.

solid answer

~50 s

On a handheld the platform version is not something the team controls. Users decide whether to accept an update, and a large share of devices eventually reach an age where the vendor stops shipping platform updates at all, so they are frozen on an old line until the hardware is replaced. The result is an installed base spread across several version lines for years — unlike a service you redeploy or a client that updates itself quietly. So the target list carries **more than one version line as a first-class row**: the newest shipped line, the line the largest share of real usage runs, and the oldest line the product still supports. Behaviour genuinely differs across them — permission prompts, background execution limits, notification handling and system dialogs all change between lines — so a pass on the newest line is not evidence about the others.

code

yaml · 12 lines
yaml
target_list:
  version_rows:
    - line: "v9"          # oldest the product still supports
      usage_share: 0.11
      role: floor
    - line: "v11"         # the largest share of real usage
      usage_share: 0.44
      role: primary
    - line: "v12"         # newest shipped
      usage_share: 0.28
      role: leading_edge
  rule: "every core case must have executed on floor and primary, not primary alone"

go deeper

for a junior

Be ready to say why a handheld app cannot be checked on one platform version alone: the user decides whether to accept an update, and some devices will never be offered another one.

for a middle

Explain the mechanics — how an installed base spreads across version lines and stays spread, which rows a target list therefore names, and which behaviours actually change between lines rather than only the version number.

for a senior

Show you have operated this: real usage share driving which lines the suite runs, the version line visible on every result, and an oldest-supported row that carries the same cases as the newest rather than a reduced set.

for a principal

Own the exposure argument. Every extra version row costs run time, and leaving a line unexercised is a bet on behaviour you have not observed; be able to defend the shape of the list with field evidence rather than with habit.

A handheld product's user base does not sit on one platform version, and it never will. Three separate mechanisms hold that spread open, and none of them is under the team's control. Understanding them is what turns a target list from a habit into a decision. ## Why the installed base will not move - **The update is the user's decision.** On a handheld the platform update is offered, not applied. It is large, it wants power and a good connection, it takes the device out of use while it installs, and a meaningful share of people decline it indefinitely — often because a past update left their device feeling slower. - **Hardware runs out of updates.** Every device has a support window. Once it closes, the device is frozen on whatever line it last received. It never becomes current again; it leaves the field only when it is replaced, and inexpensive hardware is kept for a long time. - **The path to a device is not direct.** Platform releases are frequently rebuilt and staged for particular device families by intermediaries, so two devices bought in the same week can be offered the same line months apart, and some are never offered it. The effect is cumulative. Each year adds another cohort of devices that stall where they stopped, so the longer a product ships, the wider its tail grows and the smaller the share of real usage the newest line represents. ## This is unlike most software you deploy | Where the code runs | Who chooses the version | How long an old version survives | | --- | --- | --- | | A service the team operates | the team | until the rollout finishes | | A client that updates itself | the client, usually silently | days to weeks | | A handheld app's underlying platform | the user, bounded by the hardware's support window | years, sometimes the life of the device | The middle row is the one that misleads people. An engineer whose instincts were formed where old versions age out on their own carries the assumption that the tail is temporary. On a handheld it is permanent, and it is a property of the field rather than a backlog item. ## What actually differs between version lines A version row is not bureaucratic box-ticking; it is a different program environment. Across lines you will find changes to: 1. **The permission model** — when a prompt appears, how often it may be asked again, and whether a grant is permanent or lasts only while the app is in use. 2. **Background execution** — how much work an app may do once it is no longer in front of the user, and how aggressively it is stopped. 3. **Notification handling** — grouping, priority, and whether the user must opt in before anything is delivered. 4. **Storage access** — what an app may read outside its own area, and what it must ask for. 5. **System-drawn interface** — dialogs and pickers the app does not draw but must react to, whose shape and callbacks change. 6. **Accessibility and display defaults** — default text size, contrast and animation settings that shift what a layout receives. On platforms that gate behaviour changes on the version a build *declares it targets*, the same physical device can behave differently before and after that declaration is raised — so the version axis moves for two independent reasons, one of them entirely inside your own release. ## What the target list does about it - Name the lines and give each a role: the newest shipped line, the line carrying the largest share of real usage, and the oldest line the product still supports. - Treat the oldest supported line as an ordinary row, not a favour done when there is spare time. It is where the permission and background-execution differences bite hardest. - Keep the version line visible on every result, so a failure that only happens on one line is not filed as an intermittent one. - Do not let new cases land only on the newest line. A case that has never executed on the oldest supported line is not covering it, no matter what the list says. ## Common mistakes - Reasoning from the team's own devices, which skew new, fast and recently updated. - Assuming an aggressive release cadence on your side moves users — your release cadence and their platform version are unrelated. - Reading a single line's pass as coverage of the family, when the lines differ in behaviour rather than only in version number. - Planning to "catch up later", which ignores that the tail grows rather than shrinks as the product ages.

  • What kinds of behaviour actually differ between platform version lines, enough to make the same case fail on one and pass on another?
    Permission prompts and how long a grant lasts, how much work an app may do once it is in the background, notification opt-in and delivery handling, what storage an app may reach, and the system-drawn dialogs the app must react to but does not draw. Default text size and contrast settings also shift between lines, which changes what a layout receives.
  • Why is a device that has stopped receiving platform updates a different kind of row from an older device that still gets them?
    The frozen device will never move: its line is its line until the hardware is replaced, so coverage there is a permanent commitment and its share falls only through replacement. A device still receiving updates will move on its own, so a failure specific to it has a natural expiry date and may not justify a standing row.
  • Why does targeting only the newest line get riskier the longer a product has been shipping?
    Because the tail accumulates. Every year adds a cohort of devices that stall on the last line they received, so the share of real usage represented by the newest line shrinks while the untested remainder grows. A policy that was nearly adequate at launch quietly becomes a large blind spot three years later.

It is closer to shipping a physical appliance than to updating a website: once it is in someone's hand, the version it runs is their choice and their hardware's, not yours.

saying these in an interview costs you the question

  • Assumes everyone updates to the newest version quickly
  • Treats the oldest supported line as an optional extra
  • Claims a pass on one line covers the others
  • Believes a platform update can be pushed to every device
  • Picks version rows from team devices rather than the field
open as a page

A staged rollout leaves two app versions live against one backend. What must your test coverage prove about that overlap?

level: seniorimportance: must knowfreq 58%

basics

~20 s

Coverage must prove the previous app version still works against the backend the new one requires. Run both versions' suites against the same deployed service in one pass, and cover a user who updates partway through a flow.

open as a page

Why can a suite pass on the build your pipeline makes for testing and fail on the build users install?

level: juniorimportance: should knowfreq 46%

basics

~20 s

They are different artefacts. The shipped build is optimized, has unused code stripped, is signed with the release identity, drops test hooks and verbose logging, and points at production configuration — so code the test build kept alive can vanish from it.

open as a page

What does a handheld suite that only runs on current top-tier devices never see?

level: seniorimportance: should knowfreq 46%

basics

~20 s

A top-tier-only suite never sees failures caused by scarcity: the app's process reclaimed while in the background, cold starts slow enough to expose races, memory limits hit on large content, and a device slowed to shed heat.

open as a page

Your handheld target list has not changed in two years. How do you decide what it should run on now?

level: principalimportance: should knowfreq 40%

basics

~20 s

Re-cut it from field evidence rather than habit: usage-weighted share across version lines, capability tiers and screen classes, and which rows have ever caught a failure nothing else caught. Every re-cut costs comparability, so change it on a decided cadence.

open as a page

Why is screen-size class a separate coverage axis from a handheld's capability tier?

level: middleimportance: nice to knowfreq 28%

basics

~20 s

Screen size and device capability vary independently: large low-cost displays and small powerful devices both exist. Layout failures — clipped labels, controls out of reach, a keyboard covering the focused field — track the screen class, not the hardware's speed.

open as a page

A mobile app blocks its own use below a minimum supported version. How do you design cases proving the gate cannot be bypassed?

level: middleimportance: nice to knowfreq 30%

basics

~20 s

Test the boundary and the ways around it: at the minimum version, one below it, and with the version check unreachable. The decisive case asserts the service refuses a blocked client, since a screen the app draws can always be skipped.

open as a page