Messaging & Eventing

ReceiveMessage hides a message for the visibility timeout instead of removing it, giving one consumer a temporary exclusive lease. Only DeleteMessage removes it; if the lease expires first, the message becomes visible again and is delivered to another consumer.

Standard SQS queues give near-unlimited throughput with best-effort ordering and possible duplicate deliveries. FIFO queues preserve order within each MessageGroupId and deduplicate messages, but cap throughput. Choose FIFO only when ordering or deduplication is genuinely required.

maxReceiveCount caps how many times SQS may deliver one message. Every ReceiveMessage delivery increments that message's ApproximateReceiveCount — whether the consumer failed, crashed, or never answered — and once the count passes the limit, SQS routes the message to the dead-letter queue.

MessageGroupId is the ordering scope of a FIFO queue: SQS delivers messages sharing a group id strictly in order, one in flight at a time, while different groups proceed independently. The number of distinct group ids therefore sets the maximum consumer parallelism.

The visibility timeout is shorter than the handler — with the 30-second default, the lease expires while the first worker is still running, so SQS makes the message visible again and hands it to another worker. Raise the timeout above worst-case processing time, or heartbeat with ChangeMessageVisibility.

Amazon SNS pushes once and eventually drops a message it cannot deliver. Putting an SQS queue in front of each consumer stores that consumer's copy durably, so each one retries, backs off and drains at its own rate without losing events.

Attach a filter policy to the subscription. It is a JSON document of accepted values that SNS evaluates before delivery, either against the publisher's message attributes (the default) or against the JSON message body when the subscription's filter policy scope is set to the body.

A FIFO topic preserves publish order within a message group and deduplicates repeated publishes inside a short window. Publishers must supply a message group id and a deduplication id (or enable content-based deduplication), and the ordering only survives if the subscriber is a FIFO SQS queue.

Check the topic's access policy, the subscription's confirmation state and filter policy, the destination queue's resource policy allowing the SNS service principal to send, and the KMS key policy if the queue is encrypted. A successful Publish only proves you could write to the topic.

SNS retries each failed delivery according to that subscription's delivery policy and then discards the message. To preserve it, attach a redrive policy naming a dead-letter SQS queue on the subscription, and enable delivery-status logging so failures are visible.

Every account and region has one default bus, which is where AWS services emit their own events. Custom buses are ones you create for your application's events. Partner buses receive events from a SaaS provider you have associated. Application events belong on a custom bus.

An event pattern mirrors the event's JSON structure: every field named in the pattern must match, values inside an array are alternatives, and fields the pattern omits are ignored. String matches are exact and case-sensitive unless you use a content filter such as prefix or wildcard.

A PutEvents entry carries Source, DetailType and Detail, plus optional EventBusName, Resources and Time. Because PutEvents is a batch call, a 200 response can still report FailedEntryCount above zero, and those rejected entries are lost unless the publisher resends them.

EventBridge Pipes is a managed point-to-point connector: one source (SQS, Kinesis, DynamoDB Streams, MSK, Amazon MQ) to one target, with optional filtering and enrichment in between. It replaces glue Lambda code whose only job is polling, filtering and forwarding.

EventBridge Scheduler is a separate service built for scale and per-entity timers: one-time schedules, a far higher schedule quota, a per-schedule IAM role, retry and dead-letter settings per schedule, time-zone support, and flexible time windows that spread invocations.

A shard is Kinesis Data Streams' unit of capacity and ordering: roughly 1 MB/s or 1,000 records per second in, 2 MB/s of shared reads out. Records are placed by partition-key hash and are ordered only within one shard.

On-demand suits unpredictable or new workloads: AWS manages shard count and you pay per stream-hour plus data volume. Provisioned is cheaper at steady, well-understood throughput but makes you size and reshard the stream yourself.

A standard consumer polls GetRecords and shares one shard's 2 MB/s read budget with every other standard consumer. An enhanced fan-out consumer registers with the stream and gets its own 2 MB/s per shard, pushed over HTTP/2, at extra cost.

A hot shard. Kinesis enforces throughput per shard, and a low-cardinality or skewed partition key concentrates writes on one shard's hash-key range while the others idle. Fix the key distribution first; resharding alone only buys time.

The KCL keeps a DynamoDB lease table, one item per shard, holding the owning worker and the last checkpointed sequence number. Workers heartbeat to hold leases and steal stale ones, which makes processing at-least-once and puts a second billable table in the critical path.

Amazon Data Firehose is a managed delivery pipeline: it buffers records and writes them to destinations such as S3, Redshift or OpenSearch with no consumer code and no shards to manage. You give up replay, retention and sub-second latency.

Firehose batches records before writing. Two buffering hints control the flush — a buffer size in MiB and a buffer interval in seconds — and whichever is reached first triggers delivery, so a low-traffic stream waits for the interval.

Firehose retries for the configured retry duration, then writes the still-failing records to the backup S3 bucket under an error output prefix. They are not retried again automatically — recovery means reading those objects and re-ingesting them yourself.

Enable record format conversion, which reads the schema from an AWS Glue Data Catalog table and writes Parquet, and enable dynamic partitioning to build the S3 prefix from fields in each record. Both must be planned at stream creation, and partition keys must be low cardinality.

The function must return a records array with one entry per input record, each carrying the same recordId, a result of Ok, Dropped or ProcessingFailed, and base64-encoded data for Ok records. ProcessingFailed records are written to the delivery stream's S3 error output prefix.

Step Functions matches a failure against the Retry array first and re-runs the state until that retrier's MaxAttempts is used up. Only then is the error offered to Catch. IntervalSeconds, BackoffRate and MaxAttempts define the wait between attempts.

Request-response calls the API and moves on as soon as it returns. The .sync suffix makes Step Functions wait until the underlying job reaches a terminal state and returns its result. The .waitForTaskToken suffix pauses the execution until something calls SendTaskSuccess or SendTaskFailure with the injected token.

Standard workflows are durable and auditable: they run up to a year, execute exactly once, keep a queryable execution history, and bill per state transition. Express workflows run up to five minutes, are at-least-once, log to CloudWatch, and bill by requests plus duration.

States.ALL is the Step Functions wildcard error name that matches nearly every error a state can raise. It must be the only entry in its ErrorEquals array and must sit in the final retrier or catcher, because they are evaluated top to bottom.

An Amazon States Language definition requires two top-level fields: StartAt and States. StartAt names the first state to run, each state's Next names the state that follows it, and End set to true finishes that branch.

The SES sandbox is the default restricted state of every new SES account, per Region. In it you may only send to identities you have verified (plus the SES mailbox simulator), at a low daily cap and send rate. You leave it by requesting production access for that Region.

Attach an SES configuration set with an event destination — SNS, Kinesis Data Firehose, CloudWatch or EventBridge — and subscribe to the Bounce and Complaint event types. Email feedback forwarding just drops human-readable mail into a mailbox nobody parses or monitors.

It is a per-account list of addresses SES refuses to deliver to, populated automatically after a hard bounce or a complaint. A send to a suppressed address is still accepted and returns a message ID, but SES never delivers it and reports a bounce event instead.

Easy DKIM has SES generate and rotate the signing key pair — you publish three CNAME records and forget it. BYODKIM means you supply your own private key and publish the public-key TXT record yourself, keeping key custody and rotation.

An email identity verifies one exact address by emailing a confirmation link to it. A domain identity is verified by publishing DNS records, and then any address at that domain may be the From. Production services verify the domain, because addresses like no-reply@ have no mailbox to click a link.