Client Rate Quotas

## The two halves of the definition A **rate ceiling** is an allowance on how much of a broker cluster's serving capacity one named owner may consume per unit of time. Almost everything worth knowing follows from two words in that sentence: *owner* and *rate*. The first is **owner**. A rate ceiling attaches to a **quota principal** - the identity the cluster holds responsible for the traffic. Depending on the platform and on how it authenticates connections, that can be: - a client identity an application announces when it connects; - the authenticated user or service account the connection carries; - a tenant, namespace or project grouping many applications together; - a catch-all default that applies to every principal for which no specific allowance was configured. The consequence that catches people out is that the allowance does **not** attach to the connection. If one application opens forty connections to spread its traffic, and all forty present the same identity, they share one allowance between them. Adding connections cuts the same ceiling into smaller pieces; it does not buy more of it. A cap on how many connections a node will hold at once is a genuinely different control with a different unit, and confusing the two is the most common error on this subject. The second word is **rate**. An allowance of so much per second is not checked against one request in isolation; it is checked against traffic accumulated over an averaging interval. That is why a client can send a short burst well above its nominal figure and see nothing at all, and why a client that has been over for a while keeps being held back even after it has slowed down. ## What gets counted Three units are in common use, and each prices something different. | unit | what it measures | the client it catches | |---|---|---| | bytes per second | volume accepted from a writer, or served back to a reader | large payloads, or heavy batching | | requests per second | calls made, whatever their size | many tiny calls, each cheap in bytes | | request-time share | the fraction of a node's request-handling capacity used | work that is expensive to serve but small on the wire | Bytes alone is the usual starting point and the usual mistake. A principal issuing thousands of near-empty requests a second stays far under a byte ceiling while occupying a large share of the threads that actually do request work. That is precisely what a request-time share exists to price. Read and write traffic are usually held against separate allowances, because the two cost a node different things: one is an append, the other is a read that may or may not be served from memory. ## Where the allowance is evaluated Designs differ here, and the difference changes the arithmetic you should expect: 1. Some platforms evaluate the allowance **on each node independently**. A principal whose traffic spreads across ten nodes can then consume roughly ten times the configured per-node figure in aggregate, and is held back only on whichever node it happens to concentrate on. 2. Others evaluate it for the **cluster as a whole**, which requires nodes to share consumption state, and makes the figure you configure the figure you get. 3. Rented offerings often express the same idea as a purchased allowance for the whole instance, where the ceiling is part of what you pay for rather than something you set. If you cannot say which of these your cluster does, you cannot predict what a configured figure actually permits. ## What the principal experiences Once a principal is over its ceiling, the node takes one of two actions, and platforms in this class have made opposite choices: - **Throttle delay** - the request is still served, but the answer is held back for a while first, and the client is normally told how long it was held. The symptom is latency, not failure. - **Throttle refusal** - the request is rejected and the client is expected to try again later. The symptom is a visible error, and the offered traffic falls only if the client's retry behaviour makes it fall. Neither action destroys records at the broker. A delayed write is still accepted; a refused write was never accepted, so it remains the caller's to repeat. Records go missing only when something on the client side gives up or overflows, which is a separate subject. ## Why it is asked Because the ceiling surfaces mid-incident. An application team reports that the cluster got slow, the operator looks at the cluster and finds it comfortable, and nobody thinks to ask whether an allowance configured months ago has started to bind because that one application grew. Knowing that a ceiling has an owner, a unit, an evaluation scope and one of two enforcement shapes turns that conversation into a five-minute check.