An EventBridge Pipe reading from a Kinesis Data Streams source stops making progress because one record always fails downstream. Which Pipes source settings get the pipe moving again?

## Why the stall happens at all Queue sources and stream sources fail differently, and this is the distinction the question is really testing. With an SQS source, each message is independent. A message that fails becomes visible again, gets retried, and after `maxReceiveCount` attempts the queue's own redrive policy moves it to the dead-letter queue — the queue keeps flowing around it. Notice where that logic lives: **on the queue**, not on the pipe. There is no dead-letter setting in a pipe's SQS source parameters, because SQS already owns that behaviour. With a Kinesis Data Streams or DynamoDB Streams source, records in a shard are strictly ordered and the consumer tracks a position. Advancing past a batch that failed would break ordering and lose data, so by default the pipe retries the batch — and keeps retrying. One record that always throws therefore halts *its entire shard*, while other shards continue. The symptom is a rising iterator age on one shard and a partial outage that looks maddeningly selective. ## The four controls that unblock it All of these live in the pipe's source parameters for stream sources: - **`MaximumRetryAttempts`** — the number of times a failing batch is retried before it is discarded. Without a cap, retries continue until the record ages out of the stream's retention. - **`MaximumRecordAgeInSeconds`** — discard records older than this. It is the time-based twin of the retry cap and is often the more meaningful bound, because what you usually care about is "do not let the pipeline fall more than an hour behind". - **`DeadLetterConfig`** — an SQS queue or SNS topic that receives information about the discarded batch. Set this whenever you set either of the two above, or discarding becomes silent data loss. Note the shape of what arrives: for stream sources the dead-letter record identifies the failed batch and its position rather than replaying the full payloads, so your recovery path usually re-reads the stream at that position. - **`OnPartialBatchItemFailure: AUTOMATIC_BISECT`** — on failure, split the batch in two and retry each half, recursing until the poison record is isolated to a batch of one. The good records in the batch get processed; only the genuinely bad one is discarded to the DLQ. This is the setting that turns "one bad record cost us a hundred good ones" into "one bad record cost us one record". ```json { "KinesisStreamParameters": { "StartingPosition": "LATEST", "BatchSize": 100, "MaximumBatchingWindowInSeconds": 5, "MaximumRetryAttempts": 3, "MaximumRecordAgeInSeconds": 3600, "OnPartialBatchItemFailure": "AUTOMATIC_BISECT", "DeadLetterConfig": { "Arn": "arn:aws:sqs:us-east-1:111122223333:orders-pipe-dlq" } } } ``` ## The tuning knobs around them **`BatchSize` and `MaximumBatchingWindowInSeconds`** trade latency against efficiency. They also decide the blast radius of a failure: with bisection off, batch size *is* the number of records a single poison pill takes down. A very large batch with no bisection is the configuration most likely to produce a dramatic incident. **`ParallelizationFactor`** lets more than one concurrent invocation process a single shard. It raises throughput when your target is the bottleneck, but concurrency within a shard weakens the strict end-to-end ordering you may have been relying on — raise it only when the workload tolerates that. **`StartingPosition`** matters during recovery, not steady state: `TRIM_HORIZON` reprocesses everything still in retention, `LATEST` skips the backlog. Choosing `LATEST` to "clear" a stall silently abandons the backlog. ## Diagnosing before you tune Setting retries and a DLQ stops the bleeding; it does not tell you what bled. Two things help. First, the stream's iterator age metric pinpoints which shard is stuck and when it started. Second, Pipes supports execution logging to CloudWatch Logs, S3 or Data Firehose with levels OFF, ERROR, INFO and TRACE — turning the level up gives per-execution detail on where in the source, enrichment or target sequence the failure occurred, which is usually the difference between blaming the target and discovering that the enrichment is the thing timing out. ## The judgment part Retries plus a DLQ are the right default, but they are a policy statement: you have decided that falling behind is worse than dropping a record. For a clickstream that is obviously correct. For a financial ledger it is not, and the right answer is instead to make the target idempotent and tolerant so nothing needs discarding — with alerting on iterator age so a human sees the stall within minutes. Say which of those two worlds you are in before you quote the settings; that is what separates a senior answer from a configuration recital.