When a stateless function needs to remember something between separate invocations — e.g., a shopping cart or a rate-limit counter — what are the trade-offs between externalizing that state to a key-value store like DynamoDB or Redis versus an object store like S3?

## Why the choice of store matters Once a function is accepted as stateless, the state it needs has to live somewhere external and network-addressable that every instance can reach identically — the choice of where matters because the storage systems commonly used for this (DynamoDB, Redis/ElastiCache, and S3) have fundamentally different performance characteristics, data models, and cost structures, and picking the wrong one for a given kind of state produces real production pain. ## The mechanics of each store Mechanically: - **DynamoDB** is a managed NoSQL key-value/document store: you address an item by a partition key, and reads/writes to a single item are fast — typically single-digit milliseconds. It supports atomic conditional writes and atomic counters natively, which matters enormously for concurrent serverless invocations, and it supports a `TTL` attribute that auto-expires items, exactly the shape needed for session data or rate-limit windows. - **Redis** is an in-memory data store, so single-digit-millisecond latency drops to sub-millisecond, and it offers richer data structures — sorted sets, lists, atomic increment/decrement, pub/sub — useful for leaderboards or sliding-window rate limiters. - **S3**, by contrast, is an object store: you address a whole object by a key, there is no concept of partial in-place mutation of a large object, and per-request latency is higher, commonly tens of milliseconds, because it's built for massive scale and durability of arbitrarily large objects rather than hot, frequent, small reads. ## The trade-offs that follow The trade-offs follow directly from those mechanics. 1. Using DynamoDB or Redis for what should be object storage — say, storing a 50 MB generated PDF as a base64 string in a DynamoDB item — runs into DynamoDB's **400 KB item size limit** immediately, and even below that limit, it wastes an expensive, latency-optimized store on bulk-data throughput it wasn't built for. 2. Conversely, using S3 for what should be a fast key-value lookup — say, storing each user's shopping cart as a small JSON object in S3, fetched and rewritten on every cart change — adds tens of milliseconds of latency compared to single-digit milliseconds in DynamoDB, and S3 has no atomic 'increment this field' primitive, so concurrent cart updates from two tabs can silently clobber each other via a read-modify-write race. 3. Redis buys the lowest latency of the three but at the cost of operational complexity: connecting to ElastiCache typically requires the Lambda to run inside a **VPC**, adding a networking dependency, and Redis must have persistence (RDB snapshots, AOF) explicitly configured if you need it to survive a node restart, whereas DynamoDB's durability is a given. ## Failure modes in production The failure modes show up predictably in production. | The decision | What it produces | |---|---| | A team storing session tokens in S3 | Sees intermittent user-facing latency spikes under load because S3's per-request latency, while low in absolute terms, is an order of magnitude worse than DynamoDB for the request-per-page-load access pattern of session lookups | | A team storing large uploaded images directly as DynamoDB item attributes | Hits the 400 KB item-size ceiling the first time a real user uploads a normal-sized photo, and the fix has to be retrofitted under pressure | | A team using Redis without persistence configured | Loses an entire cache's worth of session state on a routine node failover, causing a mass logout event | ## The composed pattern The pattern that avoids most of this in practice is **composition rather than exclusive choice**: store the large, infrequently-mutated payload in S3, and store a small structured 'pointer' record in DynamoDB (or Redis for the hottest, most latency-sensitive slice) that holds metadata plus the S3 key — an order record in DynamoDB might hold order status, timestamps, and a reference to an S3 object containing the full generated invoice PDF. This is exactly the pattern behind real systems like Netflix's Lambda-based media-processing pipelines, which persist processing status and small metadata in DynamoDB while the large media artifacts themselves live in S3, with each store used for the access pattern it's actually optimized for.