Is binary searching a tuning parameter still right when each feasibility probe is a noisy 20-minute load test?

## The three assumptions the textbook version hides Binary search over an answer range is normally presented as free of operational context, which quietly assumes that (1) evaluating the predicate is cheap, (2) evaluating it twice gives the same answer, and (3) doing the evaluations one after another is fine. A load-test harness breaks all three at once, and the interesting judgment is which of them to buy back and at what price. ## Do the arithmetic before defending the method A candidate range two thousand wide costs about eleven halvings. At twenty minutes each, that is close to four hours of wall-clock time — and it is four hours of *serial* time, because each probe's location depends on the previous probe's verdict. If the search has to be repeated whenever the traffic mix shifts, this is not a one-off cost; it is a recurring one that someone will eventually stop paying, which is itself an argument for a cheaper predicate or a self-adjusting mechanism. ## Buy back parallelism The serial dependency is a property of the loop, not of the technique. If you have idle capacity, probe several candidates per round: `k` probes placed evenly inside the interval split it into `k + 1` parts, and the round narrows the range by that factor instead of by two. With ten workers, the number of rounds scales as the logarithm base eleven of the range rather than base two — around three rounds, an hour, instead of eleven probes and four hours. You spend far more total probe-hours for far less wall-clock, which is the right trade whenever the workers are idle anyway and the deadline is real. ## Buy back repeatability Noise is the risk that actually loses answers. Far from the boundary a load test gives a clear verdict; right at the flip point, where the search spends most of its probes, the outcome is close to a coin toss. And binary search has no memory and no recovery: each probe permanently commits to one side, so a single wrong verdict near the boundary deletes the region containing the truth, and the loop still terminates and still returns a confident-looking number. Practical defences, roughly in order of cost: - **Define the predicate statistically.** Make feasibility mean "met the target in at least four of five runs" rather than "met it once". That is a stronger, more stable boolean, at the price of multiplying probe cost. - **Repeat only where it matters.** Single runs while the interval is wide, repeats once it narrows to the region where verdicts start disagreeing. - **Stop early.** Chasing a boundary to a precision of one when the measurement's own uncertainty spans fifteen percent is buying digits that are noise. Stop when the interval is inside the uncertainty band and say so. ## Do not ship the boundary The search answers "where does feasibility fail?" — that is precisely the edge of the cliff. Traffic drifts, hardware is replaced, a dependency slows down, and the boundary moves. The value worth shipping is inside the feasible region by a margin you can defend, recorded together with the date, the traffic profile it was measured against, and the uncertainty of the measurement. The asymmetry of consequences drives the size of that margin: an over-conservative setting costs some throughput, while a setting one step past the boundary is a target breach in production. ## The organisational call The technical answer is "yes, with parallel rounds, repeated probes near the flip point, early stopping and margin". The leadership answer has more in it: - **Who re-runs this?** A parameter derived from a four-hour experiment decays silently as the system it described changes. If nobody owns re-running it, the number becomes folklore within two quarters. - **Is a constant the right artifact at all?** If the feasible region moves with traffic, a control loop that adjusts the parameter continuously against the same measurement may be worth more than the best static value — at the cost of a feedback system that itself has to be understood and bounded. - **Is the predicate cheaper somewhere else?** A simulator or a replay harness that approximates the load test in seconds changes the whole calculus: with a cheap predicate the textbook search is fine again, and the engineering effort is better spent on predicate fidelity than on search cleverness. - **What is reviewable?** A tuned constant with a recorded procedure, bounds and margin can be audited by the next person. A number in a configuration file with no provenance cannot, and it will be copied into the next service unchanged. ## When to abandon the search outright If the response curve cannot be argued monotone, if the noise band is wider than the range you care about, or if the total probe budget buys fewer than a handful of rounds, a coarse grid over a few candidate values plus explicit judgment is more honest than a search whose precision is manufactured. Binary search over an answer range earns its place by turning many candidates into few probes; when the probes are the scarce resource and the boolean is unreliable, that trade is much weaker than it looks on a whiteboard.