Why does an asynchronously invoked function need a failure destination when the function platform already retries failed events?
answer
- the caller stopped waiting long ago
- retries are bounded, not endless
- out of attempts or too old
- keep the event and its error
- alarm on anything arriving there
basics
~20 sAsynchronous retries are bounded: when they run out or the event ages past its limit, the platform discards it, and the caller left long ago. A failure destination keeps the event and its error for alarming, inspection and replay.
solid answer
~50 sWith asynchronous invocation the platform accepts the event, tells the caller "accepted" straight away and queues the event internally. If the handler fails, the platform retries a small, configurable number of times with a delay between attempts, and it also gives up on any event older than a configured **maximum event age**. When either limit is reached the event is dropped, and because the caller only ever saw "accepted", nothing upstream knows. A **failure destination** (a queue, a topic, an event bus or another function) receives a record of every event that ran out of attempts, usually with the error, the attempt count and timestamps. That turns silent loss into something you can alarm on, debug and replay once the cause is fixed. For messages a queue redelivers on its own, the catch-net is the queue's dead-letter queue, not the function's setting.
go deeper
Remember the chain: accepted, retried a few times, then dropped unless a failure destination is configured. The caller never hears about the failure.
Explain the two limits that end an event's retries, retry count and maximum event age, and what the destination record carries so the event can be replayed.
Show you place the catch-net where the retries happen, alarm on arrivals with an owner, and size retention to outlast the longest stretch nobody is watching.
Frame a failure destination as a fleet default rather than a per-team choice, and weigh its cost, alarm noise and storage, against silent loss of business events.
## What asynchronous invocation does with an event On a function platform an event source can hand an event over in two broad ways. In the **synchronous** path the caller waits for the function's result and sees any error itself. In the **asynchronous** path the platform accepts the event, answers the caller with a plain acknowledgement ("accepted") and puts the event on an internal queue. A worker inside the platform then invokes the function for that event, and the caller has already moved on. That handoff is what makes asynchronous invocation useful: an upload notification, an order-placed event or a scheduled job does not need to wait for the work. It is also what makes failure dangerous. The only party that knows the event failed is the platform. ## Where an event goes when the retries run out When the handler throws or times out on an asynchronous event, the platform does not give up at once. It retries, with a growing delay between attempts. Two settings bound that loop: - **Retry count** - how many extra attempts the platform makes after the first failure. It is configurable, usually from zero up to a small maximum. - **Maximum event age** - how long the platform keeps trying from the moment it accepted the event. Time spent waiting in the internal queue (for example while the function is throttled) counts toward it. Whichever limit is reached first ends the event's life. What happens next depends on one setting: 1. With **no failure destination**, the event is discarded. A function error metric may show that attempts failed, but the event itself and its payload are gone. 2. With a **failure destination**, the platform writes a record of the event there and then stops. So the platform's retries cover **transient** trouble: a brief network blip, a downstream service that restarts within the retry window. They do not cover an outage longer than the retry window, a bug that fails every attempt or a malformed payload. Those are exactly the cases where you most need the event back. ## What a failure destination receives The destination is a durable target the operator chooses: a queue, a pub/sub topic, an event bus or another function. The record it receives typically contains: - the **original event payload**, so it can be replayed unchanged; - **error details** from the last attempt (error type, message, sometimes the function's response); - the **attempt count** and timestamps, so you can tell a slow failure from an instant one; - the **function identity** (name and version or alias), so a shared destination can be sorted by owner. Exact fields differ between platforms, so check what yours includes before you build replay tooling on it. Some platforms also offer a **success destination** on the same path, used to chain work rather than to catch failures. ## Function-side destination vs source-side dead-letter queue The catch-net belongs where the retries happen. A function's failure destination covers events the platform invoked asynchronously. A queue that the platform polls on the function's behalf redelivers messages itself, counts how often each one was received, and moves a message to **its own dead-letter queue** after a maximum receive count. | Event reaches the function through | Who retries | Where failed events land | |---|---|---| | The platform's asynchronous invocation path | The function platform | The function's failure destination | | A queue the platform polls | The queue, by redelivery | The queue's dead-letter queue | | A pub/sub subscription that pushes to the function | The subscription's delivery policy | The subscription's dead-letter target | Configuring the wrong side leaves a gap: a destination on the function does nothing for a message that a queue keeps redelivering. ## Making the destination useful A destination nobody looks at is only a slower way to lose events. The operating habits that make it work: - **Alarm on arrivals.** One message in a failure destination means at least one event ran out of attempts. Alarm on any arrival or a non-zero depth, not on a dashboard someone might open. - **Name an owner.** The alarm goes to the team that owns the function, not to a shared inbox. - **Keep retention longer than your blind spot.** If the team can be unavailable over a weekend, the destination must keep records well past that, or they expire unseen. - **Plan the replay.** Know how a record is turned back into an event and sent through the normal path after the fix. A function error rate is not a substitute: it counts failed attempts, including ones a later retry fixed, and it never tells you which payloads were finally lost.
- What should alert on a failure destination, and why is a dashboard not enough?Alarm on any arrival or a non-zero depth, and on the age of the oldest record, routed to the function's owning team. A dashboard needs someone to look at it, and a destination's retention is finite: records that arrive on a Friday night and are noticed after they expire are lost exactly as if no destination existed.
- If a function reads from a queue the platform polls for it, does the function's failure destination catch a message that keeps failing?Usually not. The queue redelivers the message itself after each failed attempt and counts receives; once the maximum receive count is reached the queue moves it to its own dead-letter queue. That queue-side setting is the catch-net there. Check, per source, which side does the retrying and put the catch-net on that side.
A courier who tries a delivery a few times and then returns the parcel to a depot with a note saying why. Without the depot, the parcel is simply thrown away after the last attempt, and the sender, who was told "accepted for delivery", never hears about it.
- The platform keeps retrying a failed asynchronous event until it eventually succeeds.
- The caller receives the error, so it can retry the asynchronous invocation itself.
- The function's logs hold enough to rebuild any failed event's payload later.
- A function's error rate shows which events were lost, so the destination needs no alarm.
- A failure destination on the function also catches messages a polled queue keeps redelivering.