When a function platform throttles requests, why do the function's own logs stay clean, and where does the throttling show up instead?
answer
- turned away at the front door
- a separate metric, not errors
- what the caller's status code means
- backlog age for queued work
basics
~20 sA throttled request is rejected before the handler runs, so the function's logs and error count never see it. Throttling shows up in the platform's throttle metric, as too-many-requests errors at callers, and as rising backlog age for queued events.
solid answer
~50 sThrottling happens at the platform's front door: when no concurrency is available, the request is turned away and no handler code runs, so there is no log line and the function's error metric stays flat. An operator sees it in three places. The platform reports **throttled requests** apart from errors, as a throttle count or a count of too-many-requests rejections. A synchronous caller gets a too-many-requests response, usually an HTTP `429`, which means *no capacity right now*, not a bug, so the caller should retry with exponential backoff and jitter. For asynchronous and queue-driven work nothing fails visibly at first: the platform holds and retries the events, or the messages stay in the queue, so the signal is the **age of the oldest waiting event or message** climbing. A dashboard that charts only invocations, errors and duration can look healthy in the middle of a throttling incident.
go deeper
Remember that a throttled request never reaches the handler, so logs and error counts stay clean. Know the throttle metric exists and that a 429 means retry later with backoff.
Explain how throttling looks different on the synchronous, asynchronous and queue-driven paths, and why retries need exponential backoff with jitter rather than a fixed interval.
Show you can read throttles against concurrency to tell the ceiling from the ramp, and that you alarm on event or message age so quiet asynchronous throttling is caught before deadlines drop work.
Decide which throttling signals every user-facing function must alarm on by default, and how gateways pass throttles through so callers keep the retry-later meaning.
## Where throttling happens A function platform runs your code in **instances** (sandboxes that each hold the handler). The number of requests it may run at once is bounded: by a **regional concurrency quota** shared by the functions in one region of an account or project, by the **scale-up rate** that limits how fast new instances appear, and by any **per-function concurrency cap** an operator has set. When a request arrives and none of those allow another execution, the platform **throttles** it: it refuses or defers the request at the routing layer. The key point for an operator is *where* that refusal happens. It happens **before an instance is chosen and before the handler is called**. Nothing in your code runs, so: - there is no log line from the handler, because the handler never started; - the function's **error** metric does not move, because an error is something an invocation returns, and there was no invocation; - the function's **duration** chart may even look better, because the requests that did get through ran on warm instances. That is why "the logs are clean" is not evidence that a spike went well. ## Signal 1: the platform's throttle metric Function platforms generally report throttled requests apart from errors: as a throttle count beside invocations, or as requests rejected with a too-many-requests code. It is the most direct signal and the first one to put on a dashboard and an alarm. Read it beside **concurrent executions**: throttles with concurrency near the regional quota point at the ceiling; throttles with concurrency well below it point at the scale-up rate or at a per-function cap. ## Signal 2: what a synchronous caller sees A **synchronous** invocation is one where the caller waits for the result, such as an HTTP request through a gateway. When it is throttled, the caller gets an error straight away, usually an HTTP `429 Too Many Requests` or the platform's equivalent throttling error. Some platforms first hold the request briefly in the hope that an instance frees up, and only then reject it, so a throttled call can also show up as extra latency before the error. What the error tells the caller: 1. **It is about capacity, not correctness.** The request itself was fine; resending it later may succeed. 2. **The platform will not retry it for you.** On the synchronous path the retry belongs to the caller. 3. **The retry must back off.** Retrying at once, or at a fixed interval, sends every throttled client back at the same moment and makes the next wave worse. Exponential backoff with random **jitter** spreads the retries out. If the gateway in front of the function turns the throttle into a generic `500` or `502`, callers lose that meaning, so it is worth checking how the error is passed through. ## Signal 3: the age of waiting work **Asynchronous** invocations (the caller hands over an event and does not wait) and **queue-driven** ones (the platform polls a queue or stream for you) fail quietly when throttled: | Invocation type | What happens on a throttle | Where the operator sees it | |---|---|---| | Synchronous | Rejected at once (sometimes after a short hold) | Throttle metric, `429`s at the caller, gateway error rate | | Asynchronous | The platform keeps the event and retries it later, up to a maximum event age | Throttle metric, the platform's event-age or pending-events metric | | Queue or stream | The poller slows down; messages stay in the source | Age of the oldest message, queue depth, stream iterator age | In the last two rows no request is lost *yet*, so nothing turns red. The danger is the deadline: an asynchronous event that is still throttled when it reaches its maximum age is dropped or sent to a failure destination, and a queue message that waits past its retention is gone. So the useful alarm is on **age**, set well below those limits, not on errors. ## Building a dashboard that shows it A throttling-aware view of a function fleet puts these side by side: - throttle count per function, with an alarm on any sustained non-zero value for user-facing functions; - concurrent executions against the regional quota, so you can tell the ceiling from the ramp; - the caller's view: `429` rate at the gateway or client; - for event-driven functions, the oldest event or message age. With those four, the classic incident pattern, "errors flat, duration fine, customers complaining", becomes visible in the first minute instead of after the ticket queue fills up.
- Why should a client that retries a throttled call add random jitter to its backoff?Throttling hits many clients at the same moment. If they all wait the same backoff, they all return together and form a second spike that is throttled again. Random jitter spreads the retries across the backoff window, so the platform sees a smoother arrival rate it can serve as new instances come up.
- Which alarm catches throttling of queue-driven work before any message is lost?An alarm on the age of the oldest message in the queue, with a threshold well below the queue's retention period, paired with the function's throttle count. Error alarms stay silent here, because throttled messages simply wait; only their growing age shows the function is not keeping up.
- Throttled requests show up in the function's own logs with an error and a stack trace.
- A flat error rate during a traffic spike proves the function kept up.
- A too-many-requests response means the handler code has a bug.
- Throttled asynchronous events are dropped the moment they are refused.
- Retrying a throttled call immediately, in a tight loop, is the right client response.