skip to content

How do you choose a batch size and batching window for a queue-triggered function, trading message latency against invocation count?

level: middleimportance: should knowfreq 52%

answer

  1. full, timed out, or too big
  2. quiet queues wait for the window
  3. busy queues fill before it
  4. bigger batch, bigger failure unit
  5. timeout headroom for the whole batch

basics

~20 s

A batch is sent when full, when the batching window ends, or at the payload cap. Bigger batches and longer windows cut invocations but add latency on quiet queues and enlarge the unit that fails together.

solid answer

~50 s

The platform's poller collects records and invokes the function when the batch reaches the **batch size**, when the **batching window** elapses, or when the payload reaches the platform's size cap, whichever comes first. On a **busy** queue batches fill almost at once, so the window hardly matters and batch size sets the invocation count. On a **quiet** queue the window decides latency: a lone message can wait the full window. Larger batches mean fewer invocations and allow bulk writes downstream, but each invocation runs longer, needs more timeout headroom, and fails as a bigger unit unless per-item failure reporting is used. I start from the latency target to cap the window, from per-record time times batch size against the function timeout to cap the size, then measure invocations, duration and the age of the oldest message.

go deeper

for a junior

Remember the three triggers for a batch: full, window elapsed, or payload cap reached, whichever happens first.

for a middle

Explain why the window matters on quiet queues and the batch size on busy ones, and how timeout and failure unit cap the batch size.

for a senior

Show you tune from a latency target and measured per-record time, and watch oldest-message age and duration near the timeout after a change.

for a principal

Treat batch defaults as a cost and latency policy per workload class, and push teams toward source filters before they tune batches.

## How a batch is assembled When a function reads from a queue or stream, the platform runs a **poller** on its behalf. The poller reads records, groups them into a **batch**, and invokes the function once per batch. Three conditions trigger the invocation, and the first one reached wins: 1. The batch reaches the configured **batch size** (a record count). 2. The configured **batching window** (how long the poller may wait to fill a batch) elapses. 3. The batch reaches the platform's **payload size cap** for a single invocation. These are delivery settings an operator owns. They do not change what the handler does; they change how often it runs, how much it gets each time and how long a record waits before anyone looks at it. ## What each setting trades | Setting | Larger value gives | Larger value costs | |---|---|---| | Batch size | Fewer invocations; bulk writes downstream; less per-invocation overhead | Longer invocations; more timeout headroom needed; a bigger unit that fails together | | Batching window | Fuller batches on quiet queues; fewer invocations | Added latency for records that arrive when traffic is low | **Per-invocation overhead** is the reason to batch at all: every invocation pays a request charge, some fixed start-up work in the handler and often a round trip to a downstream service. Processing many records in one invocation, especially with a bulk write, spreads that cost. ## Traffic level decides which setting matters The same settings behave differently at different volumes: - On a **busy** queue, batches fill to the batch size almost at once. The window is never reached, so lengthening it changes little. Batch size is the lever. - On a **quiet** queue, batches rarely fill. The window decides when the poller gives up waiting, so a single order event at night can sit for the whole window before processing. The window is the lever, and it sets worst-case latency. - On a queue that is **busy by day and quiet by night**, a long window is invisible in daytime load tests and shows up as night-time latency complaints. ## Limits on the batch size Three things cap how big a batch should be, whatever the overhead saving: - **The function timeout.** The whole batch must finish inside one invocation. Multiply a realistic per-record time by the batch size and keep clear headroom below the timeout, or slow downstream days will kill invocations part-way through. - **The failure unit.** By default a batch succeeds or fails as a whole, so one failing record sends the whole batch back for another attempt and repeats the records that had succeeded. Per-item failure reporting changes that and is a handler concern covered with event triggers; without it, a smaller batch limits the repeated work. - **The downstream.** A bulk API may cap request size, and a big batch may hold a database transaction open for longer than you want. ## Filtering at the source Batch tuning pairs with **source filtering**: a filter rule on the subscription or event source mapping so the platform never invokes the function for events it would discard. A handler that returns early on most events still pays for every invocation and counts against concurrency. A source filter removes those events before batching, so batches hold only useful records. Remember that events which fail the filter are usually acknowledged and gone for that subscription rather than left for another consumer of it, so filter only what nothing reading that subscription needs. ## A method that holds up 1. Write down the **latency target** for the slowest acceptable record. That caps the batching window. 2. Measure **per-record processing time** under a slow downstream. With the function timeout, that caps the batch size. 3. Check the **downstream's** bulk limits and how it behaves with larger writes. 4. Deploy and watch the **invocation count**, **invocation duration** and the **age of the oldest message** in the queue. A growing oldest-message age means the function is not keeping up; rising duration close to the timeout means the batch is too big for bad days. 5. Revisit after traffic changes; settings tuned for launch volume rarely suit a quiet night or a peak season.

  • A team doubles the batching window on a busy queue and the bill does not move. Why?
    On a busy queue batches already reach the batch size before the window ends, so a longer window never comes into play. The invocation count is set by the batch size and the traffic. To cut invocations there, raise the batch size, within the timeout and failure-unit limits.
  • What happens to events that a source filter rejects?
    They are usually acknowledged and removed for that subscription; the function never sees them and they are not kept for later. That is why a filter belongs only on a subscription whose consumers will never need those events. If another consumer needs them, it should read through its own subscription.

  • A longer batching window always cuts invocations, whatever the traffic level.
  • Batch size can grow freely because the function timeout applies per record.
  • Returning early in the handler is as cheap as filtering at the source.
  • Filtered-out events stay on the subscription for another reader of it to consume later.
  • Load tests at peak volume prove the batching window adds no latency.