skip to content

In a Newman run of a Postman collection, what happens to an item's saved examples?

level: seniorimportance: should knowfreq 38%

answer

  1. The runner sends the item's request
  2. Neither read nor rewritten by a run
  3. The reply lands on the execution instead
  4. Green run proves nothing about them
  5. No record mode appends an example

basics

~20 s

Nothing. The run sends the item's own request and never consults its saved examples, and the live reply is not written back into that array. Saved examples are inert documentation that no run validates or refreshes.

solid answer

~40 s

A collection run drives `item.request`; the `response` array beside it is never read. No code path in Newman's library or the runtime it drives reads an item's stored responses, and the runtime explicitly declines to append the live reply to that array — the reply is attached to the execution record instead, where scripts and reporters see it. Two consequences follow. A green run says nothing about whether the saved examples are still accurate: they can be badly out of date while every assertion passes. And re-running never refreshes them, because there is no record mode that appends or updates an example. Keeping them current is a manual, committed act. Serving stored examples back to a caller is a separate arrangement outside the run.

go deeper

for a junior

Be ready to say that a headless collection run sends the request stored on the item, and that the saved examples sitting beside it are documentation the run neither sends nor updates.

for a middle

Explain both directions of the non-relationship: the run does not read the response array to pick or check anything, and it does not write the reply it received back into that array either.

for a senior

Demonstrate the pipeline judgment — a green run is no evidence about example accuracy, so shape-checking belongs in the item's test script while examples get documentation-grade review in diffs.

for a principal

Own where an API's reference examples should live at all, given that a collection's examples are unverified copies, and decide what the organisation commits to keeping accurate versus generating.

## What a run actually touches When a Postman collection is executed headlessly with Newman, the runner walks the collection's items and, for each one, sends **that item's own `request`**. The `response` array beside it — the item's saved examples — plays no part. It is not read to choose a request, not read to compare against the reply, and not rewritten afterwards. This is not an inference from behaviour; it is visible in the runner's source. Neither Newman's library code nor the runtime it drives contains any read of an item's stored responses. The runtime even carries an explicit note at the point where a reply arrives, saying that the response *could* have been added to the item's set of responses and deliberately is not, because iterating them would be unnecessary work. The reply is attached to the execution instead. ## Where the live reply goes instead The distinction that matters is **execution state versus file state**: | | Saved examples (`item.response`) | The live reply | |---|---|---| | Lives in | The collection file | The run, in memory | | Produced by | A human capturing a call | The request the runner sent | | Read during a run | No | Yes, by scripts and reporters | | Survives the run | Yes, unchanged | Only via a reporter's output | During a run, the reply is what scripts see and what reporters serialise; Newman collects each one onto its run summary's execution records. None of that path goes anywhere near the item's `response` array. Re-running the collection a thousand times leaves those examples byte-identical. ## Why this matters in a pipeline Three consequences follow, and they are what a senior candidate is expected to reach: 1. **A green run does not vouch for the examples.** Every assertion can pass while every saved example documents a shape the API stopped returning long ago. The run exercised the `request`; the examples were never in the loop. 2. **Running never refreshes them.** There is no "record" mode in the run that appends or updates examples. Keeping them current is a manual, committed act — someone re-captures the call and commits the changed file. 3. **They are review material, not test material.** Because each example carries its own `originalRequest` — a full copy of a request rather than a pointer at the item's — it can drift from the item silently, and no run will surface it. ## Treating examples like the documentation they are If saved examples are inert at run time, they need the discipline you give any other checked-in documentation: - Put the collection under version control and read the `response` array in diffs, not only `request`. - Update examples in the **same commit** that changes the request they illustrate. - Keep the set small and curated. Every example is a hand-maintained copy, and a large stale set is worse than a small accurate one. - If something must actually verify the shape of a reply, write it as an assertion in the item's test script, where a run will run it. An example cannot fail. - Do not read "the collection runs green" as "the collection's examples are accurate" in a review; the two claims are unrelated. ## The boundary worth naming Candidates often jump from "examples exist" to "so they get served back to callers". Serving stored examples to a client is a **separate arrangement** — a hosted service standing in front of the collection — and it is not what a headless run does. Within the run itself, the answer to "what happens to the saved examples" is precisely **nothing**: they are neither consumed nor produced, and the only thing the runner does with an item is send the request the item declares.

  • If a saved example cannot fail, what should you write instead to verify a reply's shape?
    Put the check in the item's test script, where a run actually executes it. An assertion on the live reply fails the run when the shape changes; a saved example sits in the file and reports nothing. Examples are for a human reader, assertions are for the pipeline, and the two are not substitutes for one another.
  • Where does the reply from a request go during a run, if not into the item's `response` array?
    Onto the execution. The runtime attaches the reply to the run's context so scripts can read it, and Newman collects each one onto the execution records in its run summary, which is what reporters serialise. None of that path writes back into the collection file, so the file the run started from is unchanged when it ends.
  • How do you keep saved examples from quietly rotting in a repository?
    Treat them as checked-in documentation with an owner. Read the `response` array in collection diffs, not just the `request`; require an example to be re-captured in the same change that alters the request it illustrates; and keep the set small and curated, since every example is a hand-maintained copy and a large stale set is worse than a small accurate one.

saying these in an interview costs you the question

  • Thinks the runner sends a saved example instead of the request
  • Believes a run overwrites saved examples with the live reply
  • Says a green run proves the saved examples are current
  • Expects a record mode that appends examples during a run
  • Treats a saved example as an assertion that can fail
  • Assumes the runner compares the reply against the example