A stopwatch around a job's declared steps reports four milliseconds while the run takes forty minutes — what did it measure?

## What the four milliseconds covered The timed region contained **assembly calls** — calls that only add a step to the graph and compute nothing. Between the start and stop of that clock the engine built the **step graph**: the ordered set of steps it derives from the program, each step naming the steps whose output it reads. Depending on the engine it may also have resolved column names and types against a schema it already knows, consulted a catalog, and in some designs listed the files it will read. None of that touches a record. So the measurement is real, it is just a measurement of a different thing: *how long it takes to describe the job*. Four milliseconds is a perfectly plausible answer to that question and tells you nothing whatsoever about the forty minutes. ## Where the clock belongs Move it around the **demanding call** — the call that asks for an answer or for the output to be written, and so is what actually makes the assembled graph run. That gives a defensible wall time for the run. Be honest about what that single number now contains: - the time the engine spends rewriting the declared graph and choosing how to run it; - waiting for capacity, if the cluster is shared or the workers have to be started; - reading the input, which for a wide scan is often most of it; - every point where records must be sent between workers because the next step needs records another worker holds; - the final commit of the output, which on some storage systems is a rename and on others is a slow copy. All of it is attributed to one line of your program, and that line is usually the least interesting one. So the second measurement answers *how long did this run take* and still does not answer *what cost the time*. Which step cost what is read from the engine's own reporting about the running job, and that is a separate subject from this one. ## Timings that are not comparable A benchmark on this kind of engine goes wrong in a few reliable ways: - **Timing assembly instead of execution** — the case in the question, and the one that produces suspiciously wonderful numbers. - **Timing a demand that does less than you think.** A demand for a handful of sample rows may let the engine stop early once it has enough, so it never touches most of the input. A row count may be answerable from metadata the storage format already carries. Neither is a measurement of the full job. - **Comparing two demands over the same graph without knowing what was retained between them.** Whether a computed result is being held for reuse changes the second number completely; pinning a result for reuse is its own subject, but a benchmark that does not state whether it happened is not reproducible. - **Ignoring the first run's fixed costs.** Starting workers, warming storage-layer caches and compiling generated code land on the first run and distort a single-shot comparison. - **Treating one run as a measurement at all.** Two runs of the same job over the same input can differ substantially for reasons that have nothing to do with your change. ## Jobs with no elapsed time The question assumes a job that ends. Two common models do not: 1. **One fixed graph kept running**, with each record passing through as it arrives. The call that submits it returns almost immediately, and the job then runs until cancelled. "How long did it take" has no answer; the health of such a job is described by sustained throughput and by how far behind its input it is running. 2. **Continuous work run as a fast succession of small finite jobs.** The call that starts it likewise returns a handle rather than a duration. Each small job has a duration, but the interesting figure is again a rate, not a single elapsed time. A candidate who reaches for a stopwatch on either of those has not noticed which kind of job they are holding. And at the other extreme, a model that runs one grouping step at a time and writes every intermediate to storage barely has an assembly phase to mis-measure: submitting the job and running it are effectively the same act. ## Answering it in an interview Say what was inside the timed region — steps recorded, no records read — and why that means the number is not wrong, just about something else. Then move the clock to the demand and immediately state the limitation of the number you now have: one figure, one line, no attribution. Finish with the caveat that a continuously running job has no duration to measure, which shows you know the split between a finite job and a running one.