What is a test policy, and when is one worth writing above a test strategy?
answer
- Three layers, narrowing as they descend
- Above the strategy sits something rarer
- Consistency has no value with one team
- Outcomes hold, named tools expire
- Deviation must be legitimate and recorded
basics
~20 sA test policy is a short organisation-level statement of quality objectives and non-negotiables every team must meet - the frame a strategy obeys. It earns its cost only with several teams to align, or an external obligation.
solid answer
~50 sThree layers, narrowing as they descend. The **policy** is organisation-level and says why quality matters and what every team must do regardless of technology; it changes rarely and is usually a page or less. The **strategy** says how a given organisation or product line actually tests - levels, environments stance, automation stance, tooling, ownership. The **plan** covers one delivery. A policy is worth writing when several independent teams must be held to a common minimum, when a contract or regulator obliges the company to state its practices, or when the same cross-team argument keeps escalating. It is not worth it for one small team, where it becomes ceremony. Its costs are real: enforcement machinery, an exception queue, and ossification as teams conform to the letter. Design against that - state outcomes rather than named tools, make deviation legitimate and recorded, and put a review date on it.
go deeper
You are unlikely to be asked this. Knowing that something broader than a strategy can exist above it, and that it is organisation-wide rather than per-release, is enough.
Be able to place the three layers in order and say what each one fixes. Nobody expects you to have written a policy, but confusing it with a strategy is an easy mark to lose.
Show what it feels like to live under one: how exceptions were recorded on your team, which mandated minimum helped, and which one people satisfied in letter while the underlying quality stayed flat.
This is your question. Price the document - enforcement, exception handling, ossification - and be willing to say a small organisation should not have one at all, or that an existing policy should be withdrawn because no decision can be attributed to it.
## The three layers It helps to see the documents as a narrowing hierarchy rather than as a stack of templates. - **Policy** - organisation-wide, near-permanent, one page or less. Why quality matters here, what every team must do whatever it builds, who is accountable, and how a team may deviate. - **Strategy** - how one organisation or product line tests: which levels run, the stance on environments and data, the automation stance, tooling policy, who owns quality. - **Plan** - one delivery: scope, schedule, staffing, deliverables, risks, sign-off. Each layer is a *constraint* on the one below, not a summary of it. A strategy that merely paraphrases the policy adds nothing; a strategy that says how this product line satisfies the policy adds a great deal. ## What actually goes in a policy A usable policy is short and states only things that hold across every team: the quality objectives the organisation is pursuing, a small set of mandatory minimums, who is accountable for quality at team and organisation level, how a team records a deliberate deviation, and when the policy itself is next reviewed. Note what is missing - no tool names, no levels, no thresholds tuned to one product. The moment a policy names a specific product it starts to expire, and it hands every team a reason to argue with it on grounds that have nothing to do with quality. ## When it is worth the cost **Several independent teams.** The value of a policy is consistency, and consistency has no value with one team. Below roughly a handful of teams that already talk daily, the policy is ceremony: a 4-person team owning an airline seat-map service does not need one, and if you hand them a template they will fill it in once and never open it again. Their real policy is three lines in the team's own documentation, and that is correct. **An external obligation.** A contract, a regulator or a customer security review may require a stated position. Then a policy is cheaper than answering the same questionnaire from scratch each time. **A recurring cross-team argument.** If the same dispute escalates repeatedly - who is allowed to release without an independent check, who is accountable when something escapes - a written line settles it once. A policy earns its place by ending a specific argument, not by completing a documentation set. ## What it costs Policies are not free, and a principal-level answer prices them. - **Enforcement machinery.** An unenforced policy teaches everyone that written rules here are decorative, which is worse than having none. - **An exception queue.** Every legitimate deviation now needs a route and someone to approve it. Without one, teams deviate silently and you have lost the visibility the policy was for. - **Ossification.** Rules written for the systems of three years ago outlive them. A policy that has never changed is not stable, it is unowned. - **Conformance theatre.** Teams satisfy the letter cheaply - the check exists, the box is ticked - and the underlying quality does not move. This is the failure mode of any mandated minimum, and the reason outcome-shaped rules beat activity-shaped rules. ## Designing one that survives State outcomes rather than activities where you can: 'a defect that reaches customers gets a permanent automated check at the level that could have caught it' is enforceable and technology-neutral, while 'every team shall use the approved scenario runner' expires with the runner. Make deviation legitimate - a recorded reason and a named accepter - because the alternative is not compliance, it is invisible non-compliance. Put a review date and an owner on it. And check whether it is followed at all: if nobody can name a decision the policy changed in the last year, it is documentation cost with no return, and the honest principal move is to withdraw it rather than defend it. ## The interview frame This is a judgement question rather than a definition question, and interviews rarely turn on it - many strong engineers have worked only where the three layers were never separated. What distinguishes a good answer is the willingness to say *no policy* for a small organisation, and to describe what the document would have to buy before it was worth its enforcement cost.
- A team wants to depart from a mandatory minimum in the policy. How should that be handled?Through a recorded exception: what they are doing instead, why, for how long, and who accepted the risk. The alternative is not compliance but silent deviation, which costs you the visibility the policy existed to buy. If the same exception is granted three times, the rule is wrong and the policy should change - a queue of identical exceptions is data, not a nuisance.
- How would you tell whether an existing policy is doing anything at all?Ask what it decided recently. Can anyone name a choice that went differently because the policy said so, or an exception that was refused? Look at whether teams cite it unprompted, whether the exception route is used, and whether its text still describes systems the company operates. A policy nobody can attribute a decision to is cost without return, and withdrawing it is a legitimate outcome.
- Why should an organisation-level policy avoid naming specific tools?Because tool choices belong to the layer that knows the technology, and a named product dates the document immediately. It also invites teams to fight the policy on grounds unrelated to quality. State the outcome required - permanent checks for escaped defects, an independent view before release - and let each strategy name the classes of tool that deliver it.
saying these in an interview costs you the question
- Treats policy, strategy and plan as three words for one document
- Writes an organisation policy for a single small team
- Mandates named tools in an organisation-level policy
- Provides no legitimate route to deviate
- Assumes an approved policy changes behaviour by itself
- Never withdraws a policy nobody follows