Why does total // size undercount batches in an importer, and what is the correct ceiling-division idiom?

## The failure, in the shape it actually arrives A museum-catalogue importer pages through records in fixed batches. The batch size is not a round number - it was derived from a 92nd-percentile payload budget, so it lands on something like 500 records - and the batch count is computed as `total // size`. With 10,001 catalogue records that is 20, and 20 batches move 10,000 records. The 10,001st record is never fetched. Nothing raises. No log line fires. The import reports success, the row counts look plausible, and the defect surfaces weeks later as a handful of missing objects that correlate with nothing. This is the canonical silent truncation, and it is the reason interviewers ask about ceiling division: the bug is invisible in every test whose fixture size happens to divide evenly. ## Why it happens `//` is floored division: `a // b` is the largest integer not greater than the exact quotient. That is precisely the wrong rounding direction for "how many containers do I need to hold N things", where any leftover requires one more container. Floor answers "how many **full** batches are there"; the question asked was "how many batches are there **at all**". ## The three integer-exact idioms **Negate twice.** `-(-total // size)` is the compact idiom. Negating the dividend turns the quotient negative, flooring it rounds *away* from zero relative to the original, and negating the result flips it back - a ceiling. It works for any Python integers, needs no extra term, and is worth a one-line comment because it reads as a trick the first time you meet it. **Add size minus one.** `(total + size - 1) // size` says the same thing in a form many readers find more obvious: push the value up to the next multiple boundary before flooring. In C this idiom carries an overflow hazard on the addition; in Python ints are unbounded, so it is safe here. It does assume `total` is non-negative. **Use divmod when you also need the tail.** `full, rest = divmod(total, size)` then `batches = full + (1 if rest else 0)`. This is the version to prefer when the leftover matters anyway - sizing the last request, logging the tail, or asserting an invariant. ## The trap: math.ceil on a true division `math.ceil(total / size)` looks like the clearest expression of intent, and for modest values it is fine. But `/` is true division and always produces a `float`. Above 2**53 the integer no longer has an exact binary64 representation, so the division silently rounds and the ceiling can come out one too low - the same class of silent truncation you were trying to fix. Above roughly 1.8e308 the conversion raises `OverflowError` instead. In a catalogue importer the totals are small enough that it works; in anything counting bytes, offsets or hashes it is a latent defect. `-(-a // b)` has no such ceiling. ## Better still: do not count The most robust fix is often to stop computing a batch count at all and let the iteration express the batching: ```python for start in range(0, total, size): chunk = records[start:start + size] ``` `range` with a step already handles the partial tail correctly, and the slice at the end is simply shorter. No off-by-one is possible because no rounding decision is made. Use an explicit count only when something outside the loop genuinely needs it - a progress bar, a work-queue fan-out, a pre-allocated result table. ## Guarding the divisor Whatever form you pick, `size` reaching zero raises `ZeroDivisionError` from `//`, `%` and `divmod` alike. A batch size that comes from configuration, a query parameter or a computed budget must be validated where it enters - failing loudly at the boundary with a clear message beats a `ZeroDivisionError` from deep inside an import loop, and beats a silent fallback that quietly re-introduces the truncation. ## What a senior answer sounds like Name the rounding direction as the cause rather than describing the symptom; give the exact integer idiom and be able to justify why the double negation is a ceiling; flag the `math.ceil(a / b)` float hazard before being asked; mention `divmod` when the leftover is wanted too; and close with the test that would have caught it - a case whose total deliberately does **not** divide evenly by the batch size, asserting that every record is visited exactly once. Fixture sizes that divide evenly are how this bug survives a full test suite.