skip to content

How should an automated case assert on updates that a push connection delivers at unpredictable moments?

level: seniorimportance: should knowfreq 50%

answer

  1. Arrival time is outside your control
  2. Do not read one item at a time
  3. Something must already be accumulating
  4. Traffic from elsewhere can satisfy a match
  5. Counts and positions are not stable

basics

~20 s

Collect, then assert over the collection. Attach a receiver at subscription time that appends every delivery to a buffer the case owns, then assert that the buffer eventually holds one matching a correlation identifier the case injected.

solid answer

~50 s

Never read one delivery at a time and never assert on the next update. By the time the assertion runs, the wanted item may already have arrived, or three unrelated ones may sit ahead of it. Instead attach a receiver the moment the subscription is registered; it appends every delivery, with its arrival time, into a buffer owned by that case. The assertion is then a **predicate over the accumulated buffer** - does it contain an item whose correlation identifier matches the one the action returned, and whose fields say what the behaviour promises - re-evaluated until it holds or the deadline policy gives up. Assert on the properties of the matched item, never on how many items arrived or which position one occupies, because arrival counts are not stable. On failure, attach the whole buffer as the artefact.

code

pseudocode · 12 lines
pseudocode
received = []
channel.on_message(m -> received.append({ at: now(), body: m }))

correlation = trigger_action().correlation_id

matched = await until(deadline_policy, () ->
    first(r in received
          where r.body.correlation_id == correlation
            and r.body.status == "settled"))

assert matched.body.amount == 4200        # assert the item, not the traffic
on_failure: attach_artifact("deliveries", received)

go deeper

for a junior

Know that updates on a push connection do not wait for your assertion. Something has to be collecting them from the moment you subscribe, or the one you care about can arrive and be gone.

for a middle

Explain why the assertion is a predicate over everything collected rather than a read of the next item, and why the item you match must be found by an identifier your own case put into the data.

for a senior

Show the failure modes you design against: unrelated traffic satisfying a match, counts that are not reproducible under load, and a failure that arrives with no record of what was actually received.

for a principal

Own the convention across the suite - every case injects a correlation identifier, every predicate filters on it, every failure attaches its buffer - so a false green under parallel load becomes structurally impossible rather than a review habit.

## What "arrives at any moment" actually breaks Three assumptions carried over from request-and-response testing fail on a push connection. - **That the wanted update is next.** It may already have arrived while the triggering call was still returning, or it may sit behind three unrelated items. - **That arriving once means arriving once.** Redelivery, corrected versions and overlapping producers all put more than one item in front of you. - **That nothing else is talking.** Under a parallel run, background jobs and neighbouring cases push traffic onto connections watching the same category of thing. An assertion that reads one item off the connection and inspects it is therefore an assertion about scheduling, not about behaviour. ## Collect first, assert over the collection The shape that survives has three parts. 1. **A collector attached at subscription time.** A receiver appends every delivery - with an arrival timestamp - into a buffer owned by this case and by nothing else. It is live from before the triggering action until the case's cleanup detaches it. Nothing is read destructively; the buffer only grows. 2. **A predicate over the buffer.** The assertion asks *does what I have collected now contain an item matching this description*, and is re-evaluated until it holds or the case's deadline policy gives up. Because the buffer is cumulative, an item that arrived early is still there when the predicate first runs - which is exactly the failure that reading the next item cannot survive. 3. **A correlation identifier the case injected.** The predicate matches on an identifier the case itself created: the order it placed, the account it registered, a distinctive value it wrote into the request. No other case's traffic can then satisfy it. ## Correlate, do not count | Assertion shape | Holds under load? | Why | |---|---|---| | An item carrying my identifier arrived and its fields are right | Yes | Depends only on data this case created | | Exactly three items arrived on the connection | No | Redelivery and unrelated traffic move the count | | The first item after the action is the confirmation | No | Arrival order is not the case's to control | | The newest item on the connection has the settled status | No | A neighbouring case's item can be the newest | | Nothing of the cancelled kind arrived in the eight seconds observed | Weak but honest | True only of the stated window, and says so | The row that catches teams out is the fourth. Matching the newest delivery reads naturally, and it works perfectly on a developer machine running one case at a time. It fails only at full parallel width, only intermittently, and in the direction of a false green - the hardest defect in this whole area to find, because nothing is ever red. ## Absence is a different claim "No cancellation was pushed" cannot be established by observation at all; it can only be observed *over a window*. So build the window into the check: after the action, observe for a bounded period, then assert that nothing matching arrived within it, and put the window into the assertion message so a future reader knows what was actually proved. Treat the result as weak evidence - it can fail honestly, but it can never establish the general claim - and always pair it with a positive check that the expected path did happen. An absence check standing alone is satisfied just as well by a connection that was never delivering anything. ## Make the failure legible When the predicate never holds, the case fails with the least informative message in testing: a deadline expired. Fix that at the moment you design the assertion. - **Attach the whole buffer as a failure artefact**, in arrival order with timestamps. The difference between "nothing arrived" and "eleven items arrived and none matched" is the entire diagnosis, and it is free to capture. - **State the predicate in the message**: which identifier was being looked for, in which field, with which expected value. - **Record the count seen**, so a reader can tell a dead connection from a live one carrying the wrong thing. ## Assert the item, never the traffic Once the predicate has matched, assert on the matched item's own fields - amounts, statuses, identifiers, the shape of the payload. Do not turn the surrounding traffic into part of the check. How many items arrived, in what order, and whether any duplicates appeared are properties of the transport and the wider system, not of the behaviour this case is about; folding them in gives the case two subjects and makes it fail for reasons that have nothing to do with what it was written to protect. One deliberate omission: how long to observe and how often to re-evaluate the predicate is a separate design decision with its own tradeoffs. This shape assumes some deadline policy exists and asks only that the predicate be evaluated against everything collected rather than against a single read of the connection.

  • Why is asserting that the buffer holds exactly one matching delivery a fragile check?
    Delivery counts are not under the case's control: a transport may deliver the same item more than once, the producer may emit a corrected version, and unrelated traffic may share the connection. Assert that at least one delivery matches and that its fields are right. If exactly-once behaviour is itself the requirement, that is a separate case with its own design.
  • How do you write a check that no update of a given kind is pushed?
    Absence is only ever true over a stated window, so build the window into the check: after the action, observe for a bounded period and assert nothing matching arrived within it, naming the window in the message. Treat it as weak evidence - it can fail honestly but never establish the general claim - and always pair it with a positive check that the expected path happened.
  • Two cases run in parallel over connections watching the same entity type. What keeps their assertions apart?
    A correlation identifier each case injects into the data it creates, carried through to the delivery, and a filter on that identifier in every predicate. Without it a case can be satisfied by its neighbour's update - a false green that reproduces only at full parallel width, which makes it the hardest failure in this area to chase.

saying these in an interview costs you the question

  • Reads the next delivery and asserts on it directly
  • Asserts on how many updates arrived
  • Pauses for a fixed period then checks the buffer once
  • Matches the newest delivery regardless of its origin
  • Discards the buffer instead of attaching it to the failure
  • Treats absence over an unstated window as proof