After a long step.sleep(), a Cloudflare onboarding Workflow sent a second welcome email and a later step read an empty array; why, and how do you fix it?
answer
- run() is not one long call
- what the engine persists
- replay from the first line
- step names are cache keys
- return state, don't accumulate it
basics
~20 sWhen an instance wakes or the engine restarts, Workflows replays run() from the top: finished step.do() calls return their saved results, but code between steps runs again and in-memory variables start empty. Put side effects inside steps and build state from step returns.
solid answer
~50 sA Workflow instance is not one long-lived call. During a long sleep, a retry delay or a `waitForEvent()` the engine may hibernate the instance, and after a hibernation, a pause and resume or an engine restart it calls `run()` again from its first line. Each `step.do()` it reaches looks up its result by step name, plus how many times that name has been used, so a finished step returns its persisted value without running its callback again. Code written between steps does run again: an email send, a `create()` of another instance, `Math.random()`, a `console.log()`. An array that callbacks pushed into is rebuilt empty on the new pass. The fix is to put every side effect inside a `step.do()` and make it idempotent, build state only from step return values and `event.payload`, name steps deterministically, and wrap any non-deterministic choice in its own step.
code
javascript · 39 linesimport { WorkflowEntrypoint } from "cloudflare:workers";
// Stand-in for an email provider call that deduplicates on idempotencyKey.
async function sendWelcomeEmail(to, idempotencyKey) {
return { to, idempotencyKey, acceptedAt: Date.now() };
}
export class OnboardingWorkflow extends WorkflowEntrypoint {
async run(event, step) {
// Buggy version, for contrast:
// const sent = [];
// await sendWelcomeEmail(event.payload.email, "x"); // runs again on every replay
// await step.do("record send", async () => { sent.push(event.payload.email); });
// await step.sleep("wait before nudge", "1 day");
// ...on the next pass, sent is [] again
const welcome = await step.do("send welcome email", async () => {
return await sendWelcomeEmail(
event.payload.email,
`welcome-${event.instanceId}`,
);
});
await step.sleep("wait before nudge", "1 day");
return await step.do("send verification nudge", async () => {
return { nudged: welcome.to };
});
}
}
export default {
async fetch(request, env) {
const instance = await env.ONBOARDING.create({
params: { email: "[email protected]" },
});
return Response.json({ id: instance.id, status: await instance.status() });
},
};go deeper
Remember that only step.do() results are saved, and that code outside a step can run more than once.
Explain the replay: after a sleep, retry or wait, run() starts again from the top, cached steps return stored results, and in-memory variables start empty.
Diagnose duplicated side effects and lost state from the replay model, then fix them: side effects in idempotent steps, state from step returns, deterministic step names and branches.
Make the replay rules a code-review standard for Workflow code: no side effects between steps, deterministic names, idempotency keys on external calls, and a test that forces a pause and resume.
## The replay model **Cloudflare Workflows** does not keep a `run()` call alive for the life of an instance. The engine executes `run()` until there is nothing left to do right now (a sleep, a retry delay, a wait for an event), may then **hibernate** the instance and drop all of its in-memory state, and later calls `run()` **again from the first line**. A new pass also happens after a pause and resume, a restart, or an internal engine restart. On each pass, every `step.do()` the code reaches first checks the instance's persisted state: - If a step with that **name** (and that occurrence count of the name) already completed, `step.do()` returns the stored result and **does not run the callback**. - If not, the callback runs and its return value is persisted once it succeeds. So the step results are durable, and everything else is recomputed by re-executing your code. ## What survives a replay | Item | Survives a new pass? | |---|---| | A value returned from a completed `step.do()` | Yes, read back from storage | | `event.payload` | Yes, it is the immutable input of the instance | | A local assigned from a step's return value | Yes, because it is reassigned from the stored result | | An array declared in `run()` that a step callback mutated | No, it is rebuilt empty and the cached callback does not refill it | | A side effect written between steps | Runs again on every pass | | A `console.log()` outside steps | Prints again on every pass | | A value from `Math.random()` or `Date.now()` outside a step | Differs on every pass | ## Diagnosing the onboarding bug 1. The welcome email was sent by code **between** steps, so the replay after the sleep sent it again. A send inside `step.do()` would have returned its cached receipt instead. 2. The later step read an array that an earlier step's callback had `push()`ed into. On the new pass the array was declared empty again, and the earlier step was cached, so its callback never ran to refill it. 3. Neither shows up in a quick local test that never hibernates, which is why it tends to surface in production after a long `step.sleep()`. ## The fix - Move every side effect, such as the email send or a `create()` call on another Workflow binding, into its own `step.do()`. - Return the data a later step needs (a receipt, a list of IDs) and assign it from the `step.do()` result. Do not accumulate state by mutating variables. - Wrap non-deterministic decisions, such as a random A/B choice or the current time, in a step, so the replay reads the same value back. - Branch only on `event.payload` and step return values, never on values computed fresh outside a step. - Keep connections and other **non-serializable** resources out of step results. The docs place a resource such as a database connection outside steps, where it is simply rebuilt on each pass, with one exception: a Hyperdrive-backed connection is created inside each step that uses it. ## Step names are cache keys Because the lookup is by name, a step's name must be **deterministic**. A name like `` `send email at ${Date.now()}` `` never matches on the next pass, so the step runs again. A loop that names steps from a value returned by an earlier step (`` `remind ${userId}` ``) is fine, because the names come out the same each time. Step names are capped at 256 characters. For the same reason, `Promise.race()` across steps belongs inside a wrapping `step.do()`: otherwise the winner recorded on one pass can differ from the one the code acted on. ## Idempotency inside the step Caching protects a step only **after** its callback returns. If the email provider accepts the message and the attempt then times out or the engine restarts before the result is saved, the step runs again. So the body itself must be idempotent: pass an idempotency key derived from `event.instanceId`, or check whether the work was already done before doing it. The rule of thumb: a step body may run **more than once** before its result is saved, and every line outside a step runs **on every pass**.
- If completed steps are cached, why must the body of a step still be idempotent?The cache records a step only after its callback returns. If the provider commits the send and the attempt then times out, throws, or the engine restarts before the result is saved, the step runs again. So check whether the work was already done, or pass an idempotency key, such as one derived from `event.instanceId`, that the provider deduplicates on.
- Can step names be built dynamically, for example one step per item in a list?Yes, as long as the names come out the same on every pass. Loop over a value returned by an earlier step and name each step from its item, such as `remind ${userId}`. What breaks caching is a name built from `Date.now()`, randomness or an unordered source: the next pass looks up a name that was never stored, so that step runs again.
Like a cook who, after any interruption, rereads the recipe from the top: numbered steps already ticked off are skipped and their results are waiting on the counter, but any instruction scribbled between the numbered steps, such as phoning the guests, gets done again.
- run() executes exactly once per instance, so code between steps runs once.
- An array filled inside a step callback still holds its values after a long sleep.
- Wrapping the email send in step.do() guarantees it is delivered exactly once.
- Putting Date.now() in a step name keeps each pass's cache entry unique and safe.
- Assigning to event.payload is an easy way to hand data to later steps.
- A console.log() between steps prints once per instance.