skip to content

On a hosted browser fleet, what must your harness record to measure consumption?

level: middleimportance: should knowfreq 44%

answer

  1. measure the thing you control
  2. a row for every session opened
  3. both ends of every session
  4. the worker count is not the width
  5. a row with no close is the leak

basics

~20 s

Record a row for every session your suite opens: when you asked, when it became usable, when you closed it, how it closed, and which case and job it served. Duration comes from the timestamps; concurrency comes from overlapping them.

solid answer

~50 s

Instrument the thing you control — the session. For every session the suite opens, write a row carrying the identifier, the moment you asked for it, the moment it became usable, the moment you closed it, how it closed, and which case and job it served. Duration is then arithmetic rather than estimate. **Concurrency is not your worker count**: workers idle between cases, teardown overlaps the next open, and retries hold a replacement beside the session it replaces, so derive the peak you actually reached by sweeping the open and close moments for overlap. Keep the wait for a slot in its own column, because whether a provider meters that wait is a rule you establish rather than observe. Then reconcile your totals against the vendor's record — a gap means either a session you never closed or a meter counting something your rows cannot see.

code

python · 14 lines
python
# Every session the suite opens leaves a row, closed or not.
def open_tracked(case_id, job_id):
    asked_at = clock()
    handle = fleet.open_session(case_id)
    row = start_row(session=handle.id, case=case_id, job=job_id,
                    asked_at=asked_at, usable_at=clock())
    return handle, row

def close_tracked(handle, row, outcome):
    fleet.close_session(handle.id)
    finish_row(row, closed_at=clock(), outcome=outcome)

# Held time is closed_at minus usable_at. The wait is usable_at minus asked_at.
# A row that never reaches finish_row is a leak; query for those first.

go deeper

for a junior

Start by making the harness say when each session opened and when it closed. Without those two moments, every statement anyone makes about consumption is a guess wearing a number.

for a middle

Be ready to derive the peak overlap from the intervals and to explain why it differs, in both directions, from the worker count that was configured.

for a senior

Expect to be asked what you do when your rows and the vendor's record disagree. Name both explanations — a session you never closed, and a meter counting something you cannot see — and say which you check first.

for a principal

You will be asked to make consumption attributable across many suites. Argue for one row schema emitted by every harness, and for reconciliation as a standing check rather than an investigation launched after a surprise.

## Measure the thing you control The vendor's record is a bill. A bill arrives late, groups by nothing you care about, and answers no question of the form *which of our suites did that*. Your harness stands at both ends of every session it opens: it knows the moment it asked, the moment the browser answered, the moment it closed, and which case was running. Instrument that, and consumption becomes arithmetic on rows you own rather than an argument about a summary. Take the suite for a council planning-application tracker: cases that submit an application and wait for validation, cases that walk an officer's case-note trail, cases that post a public comment. Their durations differ wildly, and once every session leaves a row you can say so with a query rather than an impression. ## The row For every session the suite opens, emit a structured record carrying: - the session identifier as the fleet named it, so a row can be matched against the vendor's record; - the moment you asked for the session; - the moment it became usable; - the moment you closed it; - how it closed — passed, failed, errored, abandoned; - the case, the suite, the job or pipeline run, and the shard. Emit it as a row, not a log line. You want to group by case, by job and by outcome, and a free-text line makes each of those a parsing exercise. And emit a row for sessions that never close, by writing the opening part immediately rather than writing the whole record at teardown — a row with an open time and no close time is the most valuable entry in the table. ## Duration is arithmetic; concurrency is a sweep Duration falls straight out of the row: the close moment minus the usable moment. Nothing is estimated. Concurrency does not, and this is where most instrumentation quietly goes wrong. **The worker count you configured is a setting, not a measurement.** The width the run actually reached is a property of the intervals, and you get it by sweeping them: build a list of every open moment and every close moment, sort it, walk it keeping a running tally of how many sessions are live, and take the maximum that tally reaches. The swept peak drifts from the configured worker count in both directions: - **above it**, when a worker's teardown is still closing a session while the next worker is already opening another, when a retry holds its replacement open beside the session it replaces, or when a suite opens a session for setup beside its case sessions; - **below it**, when workers sit idle waiting on data, when uneven case lengths leave late workers with nothing, or when some workers never receive a case at all. Both directions matter. The upward drift is the one that collides with a ceiling at the moment the run is busiest. ## Keep the wait in its own column Between asking for a session and being handed one there can be a wait. Put it in its own column rather than folding it into the session's held time, for a reason about honesty rather than tidiness: **whether a provider meters that wait is a vendor rule you establish, not something your harness can observe.** Keeping it separate means you can apply either answer once you have it, without re-instrumenting anything. What a suite should actually do while it waits is a separate subject with its own owner. ## Reconciling your rows against the vendor's record Once both exist, compare them. A disagreement is information, not an error: | what you observe | what it usually means | |---|---| | your total below the vendor's | a session you never closed, or a meter counting something your rows do not — the wait, a teardown grace, a reserved window | | your total above the vendor's | a session that never really started, a retry the vendor merges and your rows keep apart, or a rounding rule | | your swept peak above the ceiling you believed you had | overlap at the seams, most often teardown against the next open | Run your own leak query before taking anything to the vendor. The ambiguity is the point: two explanations exist, and knowing which one applies is what turns a bill into a fact. ## The rows that pay for themselves - Rows with an open and no close: your leaks, listed. - Rows grouped by outcome: how much consumption went to sessions that were abandoned rather than finished. - Rows grouped by job: which pipeline caused a rise, rather than that a rise happened. - Rows grouped by case: the slowest cases, ranked by consumption rather than by irritation. - Swept peak per run: whether you are approaching a ceiling before it starts refusing you. ## What not to do - Do not read a figure off the vendor's console and call it measured — you cannot group or query it. - Do not use the worker count as concurrency; sweep the intervals instead. - Do not guess the meter's rules from the shape of a bill, and do not reason about the provider's own costs, which a tenant cannot observe. - Do not record only the sessions that closed cleanly. The interesting rows are the ones that did not.

  • How can the swept peak from your rows differ from the worker count you configured, in both directions?
    Upward, when sessions overlap at the seams — a teardown still closing while the next opens, a retry holding its replacement beside the session it replaces, a setup session beside a case session. Downward, when workers sit idle waiting on data, when uneven case lengths leave late workers with nothing, or when some workers never receive a case. The peak is a property of the intervals, not of the setting.
  • What should a session row carry so consumption can be attributed to a cause rather than a total?
    The identifiers that let you group: the case, the suite, the job or pipeline run, the shard, and the outcome the session ended on. A total tells you consumption rose; the grouping tells you it rose in one suite's retries, or in one shard, or in abandoned sessions. Keep the schema identical across harnesses so rows from different suites can be pooled and compared.
  • Your totals and the vendor's record disagree. What are the two explanations, and how do you tell them apart?
    Either you left sessions open that your rows never closed, or the meter counts something your rows do not — the wait for a slot, a teardown grace, a rounding unit. Query your own rows for opens without closes first, because that explanation is entirely yours to check. If the leak query comes back empty, the difference is a meter rule, and that is a question for the vendor rather than a calculation.

saying these in an interview costs you the question

  • Reads a figure off the provider's console and calls it measured
  • Uses the configured worker count as the concurrency reached
  • Logs a free-text line instead of a queryable row
  • Folds the wait for a slot into the session's held time
  • Records only the sessions that closed cleanly
  • Never reconciles own totals against the vendor's record