What should a suite send to a hosted browser provider alongside each session's verdict, and why?

## Why the verdict alone is not enough An annotation that says only *failed* produces a row you cannot act on. The reader already suspects something failed; what they need is **which** case, **which** attempt and **roughly why**, fast enough that opening the run is a decision rather than a reflex. Two constraints shape the whole payload, and both are easy to forget because neither is technical: - you can only search, filter and group on values **you actually sent**; - the payload lands on infrastructure you do not own, so treat every annotation as final. ## A case name that is stable across runs Stable means the same case yields the same name on every run, on every machine, in every shard, in any order. Derive it from the identity your framework already computes — the class or file, the case function, and a key for the parameter row where there is one — rather than from a human title. A title gets reworded, and the moment it does you have two unrelated sets of rows for one case and no way to join them. Some rules that hold up: - **Derive, never type.** A name assembled in code changes only when the code does. - **Make it flat and boring.** Use characters that survive a URL, a search box and a chat message. - **Do not encode the environment in it.** Branch, browser and shard belong in separate values; baked into the name they fragment the case's history. - **Assume it is permanent.** Choose the scheme before the first wide run, not after. ## Something that separates this attempt from the last A stable name groups a case's history; it also makes every attempt look alike. So send a value unique to the run — the pipeline run or build identifier — and, where a case can execute more than once in a run, a marker for the attempt. Without it, a re-run after a flake and the original say the same thing about the same name and nobody can tell which is which. The useful filters come from here too: branch, shard, nightly or change-triggered. What you rarely need to re-send is anything the service already received on the session request — browser, platform, settings. Repeating it is noise you then have to keep in sync. ## A reason short enough to read in a list On failure, send the first assertion's message, trimmed, so that somebody scanning outcomes can sort a run into *the booking confirmation never appeared*, *the slot search timed out* and *the login step broke* without opening a single one. A full stack trace does not do that: too long to read in a list, and usually more about your framework than about the product. | Sent | What it lets a reader do | |---|---| | the verdict | see that something is wrong | | a stable case name | find every run of this case | | the run and attempt | tell this execution from its re-run | | a short reason | triage without opening anything | | your run's address | get from their record back to yours | ## The link home, in both directions The annotation gives you *their record, labelled with your names*. The other half is yours to keep: record the identifier the service minted for the session in your own report, because your test could not have predicted it. Then a failing case reaches whatever the far side kept for it, and their row reaches your pipeline run. Both directions are your responsibility, and each needs one thing written down. ## What not to send This is the part that gets skipped, and the part with consequences. The payload is data you hand to a third party, and a failure message assembled by your harness is an unusually good hiding place for a secret: - an exception whose message carries the endpoint address your suite connected with, credentials and all; - an authorisation token echoed into a message by a helpful client library; - a fixture shaped like a real person's record — a name, a licence number, a date of birth for a driving-test booking; - an internal hostname that says more about your estate than you would put in a public document. The habit that fixes it: build the reason from a **short, deliberate** string rather than from whatever `toString` produced, and redact it the way you redact logs. How long the provider keeps it and who else on the account can read it are separate questions — worth asking only about data that got sent, so the cheapest control is not sending it. ## A checklist worth memorising 1. **Verdict** — reduced to whatever the service accepts. 2. **Stable case name** — derived in code from the case's identity. 3. **Run and attempt** — so this execution is distinguishable from its re-run. 4. **Short reason on failure** — readable in a list, redacted like a log line. 5. **Their session identifier, kept on your side** — so the link works the other way. Send those and a stranger can triage your suite. Send the verdict alone and even you will struggle a fortnight later.