In a Cloudflare Queues email consumer whose provider sometimes returns 429, how do message.ack() and message.retry() keep one failure from re-running the batch?
answer
- decide each message, then return
- negative acknowledgement
- retry with delaySeconds
- first call wins
- attempts drives backoff
basics
~20 sCalling message.ack() after each successful send settles that message for good, and message.retry({ delaySeconds }) puts only the rate-limited one back for later, so no thrown error fails the batch and one failure no longer re-sends the emails beside it.
solid answer
~40 sWithout explicit calls a thrown error retries the whole batch, re-sending every email already delivered. So the consumer settles each message itself: `msg.ack()` right after a successful send marks it delivered whatever happens later, and on a 429 `msg.retry({ delaySeconds })` puts just that message back, delayed (up to 24 hours) by a backoff computed from `msg.attempts`. Nothing throws, so the other messages are unaffected. `batch.ackAll()` and `batch.retryAll()` do the same for every message. The first call on a message wins: `retry()` after `ack()` is ignored, and a later batch-level call does not override a per-message one. Retried messages still count toward `max_retries` (3 by default).
code
javascript · 30 linesexport default {
async queue(batch, env) {
for (const msg of batch.messages) {
const email = msg.body;
try {
if (typeof email?.to !== "string") {
// Malformed: park it on the dead-letter queue now instead of spending retries on it.
await env.EMAILS_DLQ.send({ reason: "malformed", body: email });
msg.ack();
continue;
}
const res = await fetch(env.EMAIL_API_URL, {
method: "POST",
headers: { "content-type": "application/json", "idempotency-key": email.sendId },
body: JSON.stringify(email),
});
if (res.status === 429) {
// Rate-limited: back off this message only, longer on each attempt.
msg.retry({ delaySeconds: Math.min(60 * 2 ** (msg.attempts - 1), 3600) });
} else if (res.ok) {
msg.ack();
} else {
msg.retry();
}
} catch {
msg.retry();
}
}
},
};go deeper
Name the four calls, ack(), retry(), ackAll() and retryAll(), and say which act on one message and which on the batch.
Explain negative acknowledgement, the delaySeconds option with its 24-hour ceiling, a backoff from attempts, and the first-call-wins precedence rules.
Write the loop so no error escapes it, ack right after each side effect, and note that at-least-once delivery still needs an idempotency key at the provider.
Make per-message settlement the house pattern for any consumer with per-message side effects, and reserve the implicit batch outcome for genuinely all-or-nothing work.
A **Cloudflare Queues** consumer Worker receives messages in batches through its `queue(batch, env, ctx)` handler. By default the batch succeeds or fails as one unit: a normal return acknowledges every message, and a thrown error retries every message. For a consumer that sends one email per message, through a provider that sometimes answers **HTTP 429 Too Many Requests**, that default is wrong twice over: one rate-limited send would re-send every email already delivered in the batch, and the retry would come straight back at a provider that just asked you to slow down. The per-message API fixes both. ## The four calls | Call | Applies to | Effect | |---|---|---| | `message.ack()` | one message | Marks it delivered, whatever the handler does afterwards | | `message.retry(options?)` | one message | Marks it for redelivery in a later batch, without throwing | | `batch.ackAll()` | every message | Same as a successful return, even if the handler later throws | | `batch.retryAll(options?)` | every message | Marks the whole batch for redelivery, without throwing | `retry()` is what the docs call a **negative acknowledgement**: it puts one message back while the loop carries on with the rest. Messages that get neither call are settled by the batch outcome, so they are acknowledged when the handler returns normally. ## Delaying a retry Both retry calls accept `QueueRetryOptions`, which has a single field, `delaySeconds`. A delayed message is not redelivered until that many seconds have passed, up to **24 hours**. This is the tool for a 429: rather than hammering the provider, back the one message off. Each message carries an `attempts` count that starts at 1, so a backoff can grow with it, for example `Math.min(60 * 2 ** (msg.attempts - 1), 3600)` for 60, 120, 240 seconds and so on, capped at an hour. The consumer can also set a default delay for every retried message with `retry_delay` in its Wrangler config (or `--retry-delay-secs` on `wrangler queues consumer add`); a per-message `delaySeconds` overrides that default. ## Precedence: the first call wins The calls do not stack. The rules in the Queues docs: - After `ack()` on a message, later `ack()` or `retry()` calls on it are ignored. - After `retry()` on a message, a later `ack()` is ignored. - A per-message call made first is not overridden by a later `ackAll()` or `retryAll()`; the batch-level call applies only to the messages without one. The runtime logs a warning when it ignores a call, for example a `retry()` after `ackAll()`. In practice: decide each message's fate once, then let the batch default cover the rest. ## Putting it together for the email consumer 1. Validate the message. A malformed one will never succeed, so retrying it only burns attempts; send it to the dead-letter queue through a producer binding and `ack()` it. 2. Call the provider. On success, `ack()` at once, so a later failure in the same batch does not re-send this email. 3. On a 429, `retry({ delaySeconds })` with a backoff from `attempts`. 4. On any other error, `retry()` and let `max_retries` (3 by default) and the consumer's `dead_letter_queue` bound it. The code example for this question does exactly that. Two properties make it safe: - **No throw escapes the loop.** Every outcome is an explicit call, so one bad message never fails the others. - **Retries still count.** A message marked with `retry()` comes back with a higher `attempts` value and, once it has used up `max_retries`, goes to the dead-letter queue (or is deleted, without one) like a message that failed by throwing. ## What per-message acks do not solve Queues delivers **at least once**, so an email can still be sent twice, for example if the Worker is cut off after the provider accepted the send but before `ack()` was recorded. Per-message calls shrink that window from "the whole batch" to "one message"; they do not close it. The usual answer is to generate a unique ID when the message is produced and pass it to the provider as its idempotency key, so the provider rejects the duplicate. Designing that key in general is a question for the material on idempotent consumers; here the point is that `ack()` is a delivery decision, not a guarantee of exactly-once side effects. Finally, a caution about scale: explicit retries do not count as failed invocations, so they do not make Queues scale the consumer down. If the provider keeps returning 429 under a large backlog, the lever is a cap on consumer concurrency, not more retries.
- What happens if a Queues consumer calls msg.ack() and then msg.retry() on the same message?The `retry()` is ignored and the runtime logs a warning: the first call on a message wins. The same holds the other way round, and a per-message call made first is not overridden by a later `batch.ackAll()` or `batch.retryAll()`.
- Why retry a 429 with delaySeconds instead of letting the handler throw?A throw fails the whole batch, so every email already sent in it is sent again, and the retry comes back without a deliberate pause. `retry({ delaySeconds })` touches only the rate-limited message and gives the provider room. Explicit retries also do not count as failed invocations for consumer autoscaling.
- How do retry_delay and a per-message delaySeconds interact?`retry_delay` on the consumer is the default delay for any retried message, whether it failed by a throw or by `retry()`. A `delaySeconds` passed to `retry()` or `retryAll()` takes precedence over it for those messages.
A mail sorter handling a sack of letters stamps each one delivered or return-later as it goes; one smudged address does not send the whole sack back to the depot.
- message.retry() throws, so the rest of the batch stops processing.
- batch.ackAll() after msg.retry() acknowledges the retried message as well.
- Calling msg.ack() after msg.retry() cancels the pending retry.
- A retry delay cannot be longer than the 60-second batch timeout.
- Per-message ack() guarantees each email is sent exactly once.