A regression suite in a TestRail-class case repository is defined by pointing at a folder path. What goes wrong when the folder tree is reorganised, and how do you define the suite so it survives?

## Why a path is a fragile definition A suite defined as *everything under this folder* is defined by **location**. That is a legitimate thing to say, and it is convenient: it is one click, it needs no vocabulary, and on the day it is written it is correct. The trouble is that location is the single property of a case most likely to change for reasons entirely unrelated to what the case verifies. Cases are moved when a product area is renamed, when one team's subtree is split between two teams, when somebody flattens a hierarchy that had grown too deep, or when a feature is absorbed into another. None of those events is a mistake — reorganising a tree is normal maintenance. But each of them silently changes the answer a path-defined suite gives. ## The four failure shapes 1. **Cases leave.** A subtree is split and half its cases now live under a sibling path. The suite still resolves, still runs, and now covers half of what it claims. This is the dangerous one: the run reports a pass rate over what remains and nothing marks the absence. 2. **Cases arrive.** Somebody files unrelated cases into the subtree — a scratch area, a set of exploratory charters, a batch imported from elsewhere. The suite quietly grows, run time climbs, and failures appear from cases nobody intended to gate on. 3. **New cases never join.** A new component is added under a different part of the tree. It has exactly the attributes that should make it regression material, but the path does not reach it, so it is never run. Coverage decays continuously rather than in one visible step. 4. **The path breaks outright.** The subtree is deleted or renamed and the definition resolves to nothing or errors. This is the *best* outcome, because it is loud — and it is the one people design against, which is why the silent three keep happening. ## What to select on instead Membership should be a **fact about the case**, not about its current home. In practice that is a saved query combining: - **component** — the product area, which is stable even when the tree that displays it is not - **type** — the kind of check, so a suite can exclude, say, long-running probes without needing them stored elsewhere - **priority** — the stored case's own ranking, which is what lets a tier be defined at all - an explicit **label** where a tier genuinely needs an opt-in rather than a derivation The query is re-evaluated when the run is created, so a case authored this morning with the right values joins tonight without anyone editing the suite. Membership becomes something an author can grant deliberately, in the case, at the moment they know the answer — instead of something conferred accidentally by where they filed it. ## Migrating an existing path-defined suite A reasonable sequence, and the ordering matters: 1. **Snapshot the current membership.** Whatever the path resolves to today is your baseline, however wrong it may be. 2. **Derive the intended attribute expression** and run it. Compare the two sets. 3. **Read the difference in both directions.** Cases in the path but not the query are either mis-attributed or were never meant to be in the suite. Cases in the query but not the path are the coverage you have been missing — usually the more interesting list. 4. **Fix the attributes, not the query,** wherever a case is in the wrong bucket. The point of the exercise is to make the attributes true. 5. **Cut over, keep the old path as a comparison for a release,** then delete it. Leaving both is how you end up with two definitions that disagree. ## Making the drift visible afterwards An attribute-driven suite is far more robust, but it is not immune — it drifts when authors stop setting the attributes. So instrument it: - Record the **selected case count** with each run and alert on a step change in either direction. A suite that drops by a tenth overnight is a story; a suite that drops by one case a week for a year is the same story told slowly. - Periodically **reconcile against a second axis** — for example, count cases in a component by attribute and compare with what the tree holds for that area — so a systematic mis-attribution shows up as a widening gap. - Make the attributes the suite depends on **mandatory fields**, so a new case cannot be authored without answering the question that decides its membership. ## The underlying principle A folder path is where a case has been put. An attribute is what a case is. Selection that must stay true over years should read the second, and a tree should stay free to be reorganised whenever it stops helping people find things — which is the only job it was ever good at.