What does a team actually give up by adopting the claim check pattern for a pipeline that previously sent every payload inline on the message bus?

## A trade, not a free upgrade Adopting the claim check pattern is not a free upgrade; it is a deliberate trade of one set of costs for another, and a team should be able to name both sides explicitly rather than treating it as an obviously-correct default for anything over a broker's size limit. ## Cost one: latency The most immediate cost is latency. - **Inline design:** a consumer receives everything it needs the moment the broker delivers the message. - **Claim-check design:** receiving the message is only step one -- the consumer still has to make a separate network call to storage, wait for it to complete, and handle whatever errors that call can produce, before it has anything to actually process. For a single message this might add single-digit to double-digit milliseconds depending on the storage system and region, which is negligible for asynchronous batch or background processing but can matter a great deal on a latency-sensitive request path, where an extra synchronous hop directly extends end-to-end response time. ## Cost two: operational surface The second cost is operational: the team now runs, monitors, secures, and pays for two systems instead of one. That means: - separate uptime and latency SLOs to track; - separate IAM policies and network paths to secure; - separate failure modes to build retry and circuit-breaking logic around; - separate billing dimensions to reason about. | What you pay for | How it is usually priced | | --- | --- | | Broker | Usually per-message or per-throughput-unit | | Storage | Usually per-gigabyte-stored plus per-request | A consumer's fetch-from-storage step needs its own backoff and retry policy independent of whatever redelivery behavior the broker already provides for the reference message, and the two need to be reasoned about together -- a broker redelivering a reference message every 30 seconds while storage is down for five minutes produces a very different load pattern than a naive implementation might expect. ## Cost three: consistency The third and most subtle cost is consistency. Because the payload write to storage and the reference publish to the broker are two independent operations against two independent systems, there is no built-in atomicity between them -- nothing stops a producer crash between the two steps, and nothing stops a consumer from racing ahead of a write that technically succeeded but hasn't finished propagating in an eventually-consistent storage backend. Teams need an explicit convention (write first, confirm durability, then publish) and often defensive retry logic on the consumer's fetch to absorb the rare case where the object briefly isn't visible yet. ## What you get back Against all of that, the benefit is that the broker's cost profile stays proportional to message count rather than message size. Broker throughput, memory, and replication cost scale with how much data moves through the broker itself; by keeping every message small and constant-sized regardless of the underlying payload, the broker's operating cost and its capacity headroom become predictable and decoupled from payload size entirely. Object storage, meanwhile, is usually priced per gigabyte-month plus a small per-request fee and is built to hold vastly more data far more cheaply than broker throughput capacity would cost for the same bytes -- moving a terabyte a day through `S3` costs a small fraction of what pushing a terabyte a day of large messages through a broker cluster would cost in oversized instances and cross-node replication bandwidth. ## Where the crossover point sits The practical judgment call for any given pipeline is where the crossover point actually sits, and it is not simply 'bigger than the size limit, therefore use claim check without further thought.' - **A payload just over the cap.** If a payload is 300 KB against a 256 KB `SQS` cap, compressing it (which often gets typical JSON or text payloads well under the limit) or switching to a broker with a higher or no hard cap, such as `Kafka` with an adjusted `max.message.bytes`, is usually simpler, cheaper, and avoids the second-system overhead entirely. - **A payload that is routinely large.** Claim check earns its complexity when payloads are routinely megabytes to gigabytes in size -- video, images, large documents, bulk data exports -- where no reasonable compression or broker configuration change would bring them into range, and where the throughput and cost benefits of keeping the broker payload-free clearly outweigh the added latency and operational surface. A team that reaches for claim check on every payload over a few hundred kilobytes, regardless of whether a cheaper fix would have worked, has usually made a premature architectural commitment rather than a considered trade-off.