skip to content

Background Processing

Queues move work off the request path in batches, and Workflows run multi-step jobs that survive failures for days. Interviewers ask which fits a job and how retries behave.

on this pageshow

explore

questions

16

In Cloudflare Queues, how does a producer Worker send an order event, and how does a consumer Worker receive it?

level: juniorimportance: must knowfreq 45%

answer

  1. two config arrays, one per role
  2. a binding method on env
  3. a third handler beside fetch
  4. batch.messages, each with four fields

basics

~20 s

A producer Worker binds the queue under queues.producers and calls env.ORDERS.send(body) or sendBatch(); a consumer Worker listed under queues.consumers exports a queue(batch, env, ctx) handler and reads batch.messages, each with id, timestamp, attempts and body.

solid answer

~40 s

Both sides are configured in the Wrangler file. The producer gets a binding in `queues.producers` (`binding` plus `queue`), and code calls `await env.ORDERS.send(order)` for one message or `env.ORDERS.sendBatch([{ body }])` for up to 100; the promise resolves once the message is written to disk. The consumer Worker is attached with an entry in `queues.consumers` and exports `queue(batch, env, ctx)` beside `fetch`. Cloudflare pushes messages in batches: `batch.queue` names the queue and `batch.messages` holds `Message` objects with a system-generated `id`, a `timestamp` `Date`, the `attempts` count starting at 1, and the `body`. Delivery is at-least-once and ordering is best effort, so the consumer should key its writes on a field the producer put in the body, such as the order ID.

code

json · 9 lines
json
{
  "name": "orders-ingest",
  "main": "src/index.js",
  "compatibility_date": "2026-10-10",
  "queues": {
    "producers": [{ "binding": "ORDERS", "queue": "orders" }],
    "consumers": [{ "queue": "orders" }]
  }
}

go deeper

for a junior

Recall the two config arrays, the send() and sendBatch() binding methods, and the queue(batch, env, ctx) handler with the four message fields.

for a middle

Explain that send() resolves only once the message is stored, that one queue has one active consumer, and that ordering is best effort.

for a senior

Show how at-least-once delivery shapes the consumer: deduplicate on a producer-supplied business key, never on the system-generated message.id.

for a principal

Frame when a producer and consumer should be separate Workers with separate deploys, and how body schemas are versioned so old and new consumers both read them.

## Two roles, one queue **Cloudflare Queues** is a message queue that Workers write to and read from. A Worker that writes is a **producer**; a Worker that reads is a **consumer**. One Worker can be both, and in the order-ingestion example below one API Worker enqueues order events while a separate fulfilment Worker drains them into a database. A queue is created once, for example with `wrangler queues create orders`, and then each Worker declares how it relates to that queue in its Wrangler configuration file. ## The producer side A producer needs a **binding**: an entry in the `queues.producers` array with a `binding` name (the property that appears on `env`) and the `queue` name. The binding object exposes three methods: - `send(body, options?)` writes one message. The body can be any value the structured clone algorithm supports, and it must stay under **128 KB** (1 KB here is 1,000 bytes, and about 100 bytes of internal metadata count toward it). - `sendBatch(messages, options?)` writes many at once. Each item is a `MessageSendRequest` of the form `{ body, contentType?, delaySeconds? }`; a call carries at most **100 messages** and at most **256 KB** in total. - `metrics()` returns the queue's current `backlogCount`, `backlogBytes` and oldest message timestamp. Both send methods return a promise that resolves **when the message is confirmed written to disk**, and rejects when the write fails, for example when the producer exceeds the per-queue throughput limit (5,000 messages per second) and gets a `Too Many Requests` error. Awaiting that promise before answering the client is what makes "accepted" mean "stored". ## The consumer side A consumer is a Worker with a `queue()` handler on its default export, connected to the queue by an entry in `queues.consumers` (or with `wrangler queues consumer add`). Each queue has **one active consumer**, but one consumer Worker may serve several queues and branch on `batch.queue`. Cloudflare does not invoke the handler for an empty queue; it **pushes** batches when messages are waiting. The handler receives a `MessageBatch`: | Member | What it holds | |---|---| | `batch.queue` | the name of the queue this batch came from | | `batch.messages` | a read-only array of `Message` objects | | `message.id` | a unique, system-generated ID | | `message.timestamp` | a `Date` recording when the message was sent | | `message.attempts` | how many times delivery has been attempted, starting at 1 | | `message.body` | the deserialised body the producer sent | `batch.ackAll()`, `batch.retryAll()`, `message.ack()` and `message.retry()` also live here; how they work, and what happens when the handler throws, is covered with Queues failure handling. ## Two promises the platform does not make 1. **Exactly-once delivery.** Queues delivers **at least once**: a message can, rarely, arrive twice. Because `message.id` is generated by Queues, a producer that resends after a failed call creates a second message with a new ID. The safer deduplication key is a value the producer put in the body, such as `orderId`, used as the primary key of the insert. 2. **Ordering.** The order of `batch.messages` is best effort and is not guaranteed to match publish order. A consumer that needs the latest state of an order should compare a version or timestamp carried in the body, not rely on array position. ## First mistakes to avoid - **Putting the binding in the wrong array.** `queues.producers` creates `env.ORDERS`; `queues.consumers` attaches a `queue()` handler. A producer listed only under `consumers` has no `send()` to call. - **Not awaiting `send()`.** An unawaited send can fail without the client ever hearing about it, so the API's `202` stops meaning "stored". - **Expecting a schedule.** The consumer runs when messages are waiting, not on a timer; with an empty queue it does not run at all. - **Sending values JSON cannot carry.** With the default `json` content type, a `Date` arrives as a string and a `Map` as an empty object. - **Treating a batch as a transaction.** A batch is a delivery unit, not a guarantee that its messages belong together or arrive together. ## Putting it together The configuration below declares both roles in one Worker for brevity; in production the producer and consumer are often separate Workers, each with only its own half of the `queues` block. The `queues` key is not inherited by named Wrangler environments, so each environment must declare it again. - Producer: `fetch` validates the order and awaits `env.ORDERS.send(order)`, then returns `202`. - Consumer: `queue` walks `batch.messages` and writes each `message.body` keyed on `orderId`. - Typing: in TypeScript, `Queue<Order>` on the producer and `ExportedHandler<Env, Order>` on the consumer type `message.body`. The older Service Worker form, `addEventListener('queue', ...)`, still exists but is deprecated; new consumers use the ES module `export default { queue }` handler shown here.

  • Can one consumer Worker read from several Cloudflare queues, and how does it tell their batches apart?
    Yes. Add one `queues.consumers` entry per queue to the same Worker. Each invocation still carries messages from a single queue, and `batch.queue` holds that queue's name, so the handler can `switch (batch.queue)` and route log events, order events and email jobs to different code. Each queue still has only one active consumer.
  • Why is message.id a poor key for deduplicating orders written by a Cloudflare Queues consumer?
    Queues generates `message.id` when the message is written. If a producer's `send()` fails mid-flight and the code sends again, the second copy gets a new ID even though it is the same order. A business key the producer puts in the body, such as `orderId`, stays the same across resends and redeliveries, so it makes the right primary key.

  • The consumer Worker polls the queue from its fetch handler.
  • A queue can have several active Worker consumers sharing the load.
  • Messages arrive in exactly the order the producer sent them.
  • message.attempts starts at 0 on the first delivery.
  • The queue binding goes under queues.consumers on the producer Worker.
open as a page

In Cloudflare Queues, what do max_batch_size and max_batch_timeout default to, and what does raising them save when Queues bills per message?

level: middleimportance: must knowfreq 42%

basics

~20 s

max_batch_size defaults to 10 messages (up to 100) and max_batch_timeout to 5 seconds (up to 60); a batch ships when either is reached. Queues bills per message, so bigger batches save consumer invocations and downstream writes, at a latency cost.

open as a page

In a Cloudflare Queues email consumer whose provider sometimes returns 429, how do message.ack() and message.retry() keep one failure from re-running the batch?

level: middleimportance: must knowfreq 45%

basics

~20 s

Calling message.ack() after each successful send settles that message for good, and message.retry({ delaySeconds }) puts only the rate-limited one back for later, so no thrown error fails the batch and one failure no longer re-sends the emails beside it.

open as a page

In a Cloudflare Workflow, how do step.do() retries and timeouts behave by default, and when do you throw NonRetryableError?

level: middleimportance: must knowfreq 45%

basics

~20 s

By default each step.do() retries a thrown error 5 times with a 10-second exponential backoff and a 10-minute timeout per attempt; a per-step config overrides it. Throwing NonRetryableError stops the retries at once for a permanent failure.

open as a page

After a long step.sleep(), a Cloudflare onboarding Workflow sent a second welcome email and a later step read an empty array; why, and how do you fix it?

level: seniorimportance: must knowfreq 42%

basics

~20 s

When an instance wakes or the engine restarts, Workflows replays run() from the top: finished step.do() calls return their saved results, but code between steps runs again and in-memory variables start empty. Put side effects inside steps and build state from step returns.

open as a page

In a Cloudflare Queues consumer Worker that never calls ack(), when is a delivered batch acknowledged, and what makes it retry?

level: juniorimportance: should knowfreq 35%

basics

~20 s

A Queues batch is acknowledged implicitly once queue() returns, its promise resolves and every ctx.waitUntil() promise resolves. If queue() throws or any of those promises rejects, the whole batch fails and every message in it is retried.

open as a page

In Cloudflare Workflows, how do you define a workflow and start an instance of it from a Worker?

level: juniorimportance: should knowfreq 40%

basics

~20 s

Export a class that extends WorkflowEntrypoint and implements run(event, step), declare it under workflows in the Wrangler file, then call env.BINDING.create({ id, params }) from a Worker; the instance runs its steps durably in the background.

open as a page

In Cloudflare Queues, what does the contentType option on send() change, and when does v8 matter instead of the default json?

level: middleimportance: should knowfreq 22%

basics

~20 s

contentType picks how a Cloudflare Queues message body is serialised: json (the default for compatibility dates after 2024-03-18), text, bytes or v8. json loses values such as Date and Map; v8 keeps them but only Workers consumers can decode it.

open as a page

In Cloudflare Queues, what happens to a message that uses up its max_retries, with and without a dead_letter_queue configured?

level: middleimportance: should knowfreq 40%

basics

~20 s

Once a Queues message has used up max_retries (3 by default), it is deleted permanently unless the consumer sets dead_letter_queue, in which case it is written to that queue, which needs its own consumer to be processed.

open as a page

When would you attach an HTTP pull consumer to a Cloudflare queue instead of a consumer Worker, and how does it pull and acknowledge messages?

level: middleimportance: should knowfreq 22%

basics

~20 s

Use an HTTP pull consumer when processing runs outside Workers or must set its own pace. Enable it with wrangler queues consumer http add, POST to the queue's messages/pull endpoint, then send each lease_id to messages/ack before the visibility timeout.

open as a page

In Cloudflare Workflows, how do step.sleep(), step.sleepUntil() and step.waitForEvent() pause an instance, and what happens when a wait times out?

level: middleimportance: should knowfreq 38%

basics

~10 s

step.sleep() waits a duration and step.sleepUntil() until a Date or timestamp; step.waitForEvent() waits for instance.sendEvent() with a matching type, 24 hours by default. A timed-out waitForEvent() throws and fails the instance unless caught.

open as a page

Why does a Cloudflare Queues email consumer keep hitting provider 429s during a backlog even though it retries them with delaySeconds, and how do you cap it?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Queues autoscales push consumers on backlog growth, and explicit retry() calls do not count as failed invocations, so concurrency keeps rising and more batches hit the provider at once. Set max_concurrency to what the provider can absorb.

open as a page

Why does one malformed message make a Cloudflare Queues email consumer send some welcome emails four times and then dead-letter its whole batch, and how do you fix it?

level: seniorimportance: should knowfreq 35%

basics

~20 s

A thrown error fails the whole Queues batch, so emails before the bad message are re-sent on each of four deliveries (default max_retries 3) and the batch exhausts its retries together. Fix it with per-message ack() and retry().

open as a page

An order API Worker returns 202 and calls env.ORDERS.send() inside ctx.waitUntil(); some orders never reach the Cloudflare queue. Why, and what is the fix?

level: seniorimportance: should knowfreq 28%

basics

~20 s

Inside ctx.waitUntil(), a rejected send() is never seen: the client already got 202 and Cloudflare ignores the error. Await env.ORDERS.send() before responding, since it resolves only once the message is on disk, and return an error status when it throws.

open as a page

A Cloudflare onboarding Workflow instance ended errored at its charge step; how do you inspect it and get that user's run going again?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Read instance.status() or wrangler workflows instances describe to see the error and each step's attempts, fix the cause, then call restart() on that instance, optionally from the failed step; create() with the same ID throws while the instance is retained.

open as a page

A team set delivery_delay on a Cloudflare Queues producer binding to hold order events for 60 seconds, but production consumers get them at once. Why, and how should they delay messages?

level: seniorimportance: nice to knowfreq 15%

basics

~20 s

Wrangler 4 warns that delivery_delay on queues.producers is deprecated and has no effect. Set a queue-level default with wrangler queues update orders --delivery-delay-secs=60, or pass delaySeconds to send() or sendBatch(); per-message values override the queue default.

open as a page