Concurrently merged classification results reach an ordered audit log out of queue order: how would you restore the required ordering?

## First, interrogate the requirement A log described as ordered is usually one of three different things, and they have different prices: - **Total order by arrival.** Every record must appear in the order its item entered the queue. The strongest and rarest requirement. - **Per-key order.** Decisions about the same item must appear in order; decisions about different items never interact. This is what most consumers actually need. - **Sortable.** The consumer reads the log later and can sort on a field, so the write order is irrelevant provided the field exists. If the answer is the third, attach a sequence number at the mapping step and stop. If it is the second, the fix is partitioning rather than serialising. Only the first forces the expensive mechanisms below, and asking which one it is before reaching for them is the strongest part of a good answer. ## The three mechanisms for real total order **1. Sequential concatenation.** Subscribe to the next inner call only after the previous completed. Order is a free consequence, there is no buffer, and there is nothing to tune. The cost is throughput: one element per inner-call latency, permanently. If the sustained arrival rate exceeds that, lag grows without limit and the fix is not a fix. **2. Order-preserving concurrent flattening.** Subscribe eagerly up to the bound - so the calls still overlap and the throughput benefit is retained - but hold each completed inner result and release results **in subscription order**, which is element order. What you buy is order; what you pay is: - **Memory**: results that completed early sit held until their predecessor releases. - **Head-of-line delay**: while the earliest inner call is the slowest, nothing behind it can be delivered, so tail latency for a batch is set by its worst call. The common misconception is that this is bounded by the concurrency bound. It bounds the *number of inner sources*, not the *volume* they produced: with a bound of N, up to N-1 completed inner results wait on the earliest, and if an inner source emits many elements rather than one, the held volume is larger still. **3. Resequencing downstream.** Attach a sequence number at the mapping step, let the pipeline merge freely, and reorder in a buffer in front of the log. Mechanically this is the same trade moved further along, with one extra hazard: the buffer is now yours to bound, and a gap that never fills - an element whose call failed, or was discarded - stalls the log indefinitely unless you decided in advance on a bounded wait and what to do when it expires. ## Side by side | approach | calls overlap | held state | main failure mode | |---|---|---|---| | sequential concatenation | no | none | lag grows once arrival rate exceeds 1/latency | | order-preserving concurrent flatten | yes, up to the bound | completed results awaiting predecessors | head-of-line delay behind one slow call | | resequencing buffer downstream | yes, unrestricted | out-of-order records plus gap tracking | a gap that never fills stalls the log | | partition by key | yes, across keys | none | ordering only holds within a key | ## Partitioning, the answer people miss Group the queue by item key, concatenate sequentially inside each group, and merge across groups. Within a key you get strict order because only one call for that key is ever in flight; across keys you keep as much concurrency as there are active keys. Two conditions make it work: the keys must be numerous enough to give useful concurrency, and no consumer invariant may span keys. When both hold, this recovers nearly all the throughput of a plain merge while satisfying the ordering the consumer actually asserted - and it usually makes the per-key hot spot visible too, since a key with heavy traffic now serialises against itself. ## What not to do - **Do not sort a time window and hope.** Sorting the last few seconds of results is a fix that works until a straggler exceeds the window, and then it fails silently and rarely, which is the worst failure profile available. - **Do not raise the concurrency bound.** Ordering is not a throughput problem; more overlap makes the reordering worse. - **Do not rely on latencies being similar.** They are similar in tests and not in production, which is exactly why the defect reached the log rather than a review. ## The shape of a strong answer Name the requirement first, then pick a mechanism and say its cost out loud: order for free at one call per round trip, or overlap retained at the price of held results and head-of-line delay, or a resequencer with a bounded wait and a defined behaviour for gaps. An answer that offers only sequential concatenation solves the ordering and hides the throughput consequence, and an answer that offers only a buffer without a gap policy has moved the defect rather than removed it.