skip to content

You have a CPU-bound loop with tens of millions of cheap iterations. How do you make it cancellable without crushing throughput, and why not check on every iteration?

level: seniorimportance: should knowfreq 40%

answer

  1. Check every N iterations, not every one
  2. Size N from a latency budget (few ms)
  3. Power-of-two N => mask not modulo
  4. yield is costly => periodic only
  5. Chunk heavy work, check between chunks

basics

~10 s

Check cancellation periodically, e.g. every few thousand iterations, instead of every one. Frequent checks add overhead and, for yield(), expensive rescheduling, so you balance responsiveness against throughput.

solid answer

~40 s

Each iteration here is so cheap that even a `ensureActive()`/`isActive` read per iteration can be a measurable fraction of the work, and a `yield()` per iteration is far worse because it reschedules through the dispatcher. The pattern is **periodic checking**: keep a counter and call `ensureActive()` (or `yield()` if you also need fairness) every N iterations, where N is sized so the check happens every few milliseconds — small enough that cancellation latency is acceptable, large enough that overhead is negligible. This bounds worst-case cancellation latency to roughly N iterations while keeping the hot path branch-predictable. For truly heavy compute you might also chunk the work and `withContext`/`yield` between chunks. The trade-off is responsiveness (smaller N) vs throughput (larger N); pick N from a target latency budget, not arbitrarily.

code

kotlin · 11 lines
kotlin
suspend fun crunch(total: Long) = coroutineScope {
    var acc = 0L
    var i = 0L
    val mask = 16_383L  // check every 16,384 iterations
    while (i < total) {
        if (i and mask == 0L) ensureActive()  // bounded cancel latency
        acc += i
        i++
    }
    acc
}

go deeper

for a junior

Knows to add a check but likely checks every iteration without considering cost.

for a middle

Suggests periodic checking and knows yield is costlier than isActive/ensureActive.

for a senior

Sizes N from a latency budget, uses power-of-two masking, and reasons about throughput vs responsiveness.

for a principal

Designs chunking + fairness strategy across the Default pool and ties N to measured per-iteration cost and SLOs.

## The tension Making a loop cancellable means inserting a **check point**. But check points cost time: - `isActive` / `ensureActive()` — a cheap atomic state read, but not *free*; on a loop body that itself is just an increment, a per-iteration check can be a noticeable percentage of total work. - `yield()` — much costlier: it suspends and reschedules through the dispatcher every call. So checking on **every** iteration of a tens-of-millions loop wastes throughput. ## Periodic checking pattern Check every `N` iterations: ```kotlin launch(Dispatchers.Default) { var i = 0L val n = 8_192 // power of two => cheap mask while (i < total) { if (i and (n - 1L) == 0L) ensureActive() // check every N // ... cheap work ... i++ } } ``` - Worst-case cancellation latency ≈ time to run `N` iterations. Size `N` so that is a few milliseconds — responsive enough for users, cheap enough to ignore. - Using a power-of-two `N` lets you mask (`i and (N-1)`) instead of modulo for a faster check. ## When to use yield() periodically If the loop shares a single `Dispatchers.Default` thread with other coroutines, a periodic `yield()` (instead of `ensureActive()`) both checks cancellation and prevents **starving** the others. Same periodicity logic applies; yield's higher cost reinforces *not* doing it every iteration. ## Chunking heavy work For expensive per-iteration work, split into chunks and suspend between them: ```kotlin for (chunk in chunks) { ensureActive() process(chunk) } ``` ## Choosing N from a budget - Target latency, e.g. "cancel within ~2 ms". - N ≈ targetLatency / perIterationTime. - Measure; don't guess. Re-tune if iteration cost changes. ## Pitfalls - N too large → sluggish, unresponsive cancellation. - N too small → throughput regression. - Using `%` with a non-power-of-two on the hot path adds an integer division per iteration.

  • How do you choose N?
    From a target cancellation-latency budget: N ≈ budget / per-iteration cost, then measure. Round to a power of two so you can mask instead of using modulo.
  • When would you still check every iteration?
    When iterations are expensive (e.g. milliseconds each), a per-iteration ensureActive() is negligible relative to the work, so periodic batching buys nothing.

Like a marathon runner glancing at the crowd every kilometre, not every step — frequent enough to hear 'stop', rare enough to keep pace.

saying these in an interview costs you the question

  • Insisting you must check every single iteration
  • Using yield() every iteration in a hot loop
  • Picking N with no latency reasoning
  • Using modulo with arbitrary N on a hot path
  • Ignoring fairness/starvation when sharing a thread

context

resolving links…