skip to content

A Postman collection passes in the app but fails on a build agent's terminal run — how do you diagnose it?

level: seniorimportance: should knowfreq 31%

answer

  1. Two runs, different inputs
  2. Did the call connect at all?
  3. Transport versus values
  4. Reproduce on the machine that failed

basics

~20 s

Audit the handover before the collection. The app supplies inputs implicitly from a session; a terminal run has only the file and the arguments. Split the failure by whether the call connected, then check which input was never named.

solid answer

~50 s

Start from the assumption that **an input the app supplied was never named on the command line**, not that the requests are wrong. Split the symptom first: a call that never connects points at **transport** handover, a call that returns the wrong reply points at **values** that were not passed or differ on the agent, and a correct reply with a failing assertion is usually genuinely the service. Then check the two silent traps: a **proxy does not travel at all** because no command-line option carries one, and `--ssl-client-cert` is **appended last** with an `<all_urls>` match, so a host-scoped entry already in the list wins and the flag looks inert for exactly one host. Reproduce the real command on the agent, prove reachability with a plain request from there, and verify certificate selection from the server side.

go deeper

for a junior

Be ready to say that an app run and a terminal run do not share inputs, and that the first question is which environment, file or setting the command line was never given.

for a middle

Explain the split between a call that never connected and a call that returned the wrong reply, and name what each one implicates: transport handover on one side, value stores on the other.

for a senior

Demonstrate that you reproduce the exact command on the failing machine, prove reachability independently before touching the collection, and know the two silent traps — no proxy option at all, and a command-line certificate that sits behind any host-scoped entry.

for a principal

Own the argument that implicit inputs are the real defect. Push for runs whose dependencies are declared and provable on any machine, so a green app run is never mistaken for evidence about a build agent.

## Treat it as a handover audit, not a flaky test An application run and a terminal run of the same collection are not the same run with a different front end. The app supplies inputs implicitly from a session — a selected environment, values earlier activity wrote, an accumulated cookie jar, transport settings held in the application. A terminal run has **only the file it was given and the arguments it was typed**. So when the app is green and the agent is red, the first hypothesis is not "the API is broken" and not "the tests are flaky": it is **an input the app supplied that the command never named**. ## Split the failure by where it happens The single most useful cut is whether the run got a reply at all: 1. **No connection.** The call never reaches the service, or reaches something unexpected. That points at **transport** handover — the class of inputs the app held in its settings. 2. **A reply, but the wrong one.** Authentication rejected, a 404 for an id that exists, an empty list. That points at **values** — an environment or store that was not named, or was named but holds different contents on the agent. 3. **A reply that is right, but an assertion disagrees.** That is usually genuinely the service or the data, and is the one case where the handover is probably fine. Getting this split right early stops the classic waste of re-exporting the collection over and over against a problem that never lived in the collection. ## The two traps that produce silent differences - **A proxy does not travel.** There is no command-line option for one, so a proxy the app applied stays behind. On an agent that needs it, calls fail to connect or leave the intended path entirely, and nothing in the run reports a missing setting. - **A certificate travels with the wrong precedence.** `--ssl-client-cert` is **appended to the end** of the run's certificate list with an `<all_urls>` match, and the first matching entry is the one used — so any host-scoped entry already in the list beats it. The symptom is a flag that appears to do nothing for exactly one host. Both are silent. Neither produces a warning that would appear in the run's output, which is why they survive several rounds of "let me just re-run it". ## A symptom-to-cause table worth carrying | What you see on the agent | Most likely missing piece | |---|---| | Connection never establishes to an internal host | proxying that the app was applying and no flag carries | | Server rejects the client identity for one host only | a host-scoped certificate entry outranking the flag | | Authentication fails everywhere | the value store the requests read was never named | | A request URL still carries an unresolved token | a store named, but without the key the request wants | | Only the first request passes | state the app had accumulated across a session | ## How to close it out - **Reproduce the exact command on the agent**, not an approximation on your laptop — the difference between the two machines is frequently the entire bug. - **Enumerate the inputs the requests depend on** and check each against the command, rather than reading the collection looking for something wrong with it. - **Prove transport independently** with a plain request from the agent to the same host before touching the collection at all. - **Verify certificate selection from the far side**, since the caller's own output does not say which identity it presented. - Once found, **make the input explicit in the command or explicit in the machine's setup** — a fix that lives in one engineer's shell history is the same bug again next quarter. ## What a strong answer sounds like The senior signal here is refusing to treat the two runs as equivalent. A weaker answer starts debugging the requests; a stronger one starts by listing what the application was providing that the command line is not, ranks those by the symptom in front of it, and knows without looking that one of them — the proxy — has no command-line representation at all and therefore can only be fixed on the machine.

  • Which symptom tells you the problem is transport rather than a missing value?
    Whether a reply arrived. If the connection never establishes, or lands somewhere unexpected, the missing piece is transport configuration the application held — proxying, or which client identity was presented. If a reply came back and the assertion disagrees with it, the run reached the service and the suspicion moves to the value stores the command named, or to the data on that environment.
  • Why is re-exporting the collection usually the wrong first move?
    Because the collection is rarely the difference. The exported document is identical on both machines; what differs is the session state the application supplied around it. Re-exporting costs a cycle, changes nothing, and delays the audit of which inputs the command line never named — which is where the answer almost always is.

saying these in an interview costs you the question

  • Calls the terminal run flaky instead of auditing the handover
  • Re-exports the collection repeatedly without checking the command's inputs
  • Debugs on a laptop that already sits on the right network
  • Assumes a passing app run proves the agent can reach the host
  • Trusts the flag it passed instead of checking what the server saw