A Lambda function is invoked for every message on a busy SQS queue but discards about 90% of them immediately. How does FilterCriteria on the event source mapping change that, and what happens to the messages that are filtered out?
answer
- evaluated before the invoke
- EventBridge pattern syntax
- for SQS it deletes, not skips
- filter the record, not your parsed payload
- a wrong pattern fails silently
basics
~20 sFilterCriteria puts event-pattern matching in the AWS-managed poller, so non-matching records never reach the function and are never billed as invocations. For SQS, filtered-out messages are deleted from the queue; for streams, the poller simply advances past them.
solid answer
~50 s`FilterCriteria` on the event source mapping holds one or more content-filter patterns, written in the same JSON pattern syntax EventBridge uses. The poller evaluates each record against them **before** invoking, and only matching records go into the batch, so the 90% you were throwing away no longer cost an invocation, duration or a cold start. The critical operational detail is what happens to the rejects: for SQS, Lambda **deletes** them from the queue — filtering is not a peek, it is a consume. For Kinesis and DynamoDB Streams there is nothing to delete, so the poller just checkpoints past them. Patterns are ORed, so a record matching any one of them is kept. And for SQS you normally filter on the JSON inside `body`, which means a message whose body is not valid JSON will not match a body-scoped pattern at all — a mistyped field name silently deletes the entire queue's worth of traffic.
go deeper
Know that a trigger can be configured to only pass matching records to your function, so the filtering logic does not have to live in your handler.
Be able to write a pattern for the right scope — under body for SQS — and to state plainly that non-matching SQS messages are deleted rather than left behind.
Show that you weigh the consume-not-peek behaviour before adding a filter, and that you know the failure is silent, so you verify with a sample record rather than by watching logs.
Own the routing decision one level up: whether selection belongs in a per-consumer filter, in an EventBridge rule, or in a fan-out topology, and what each choice costs in coupling and in replayability.
## The problem filtering solves A shared queue or a change stream usually carries more event types than any one consumer cares about. Without filtering, every record becomes an invocation: you pay the request charge, you pay for the duration of a function that does `if (type !== 'order_placed') return;`, you burn concurrency, and on a cold path you pay an initialisation you did not need. `FilterCriteria` moves that test out of your code and into the AWS-managed poller, where it costs you nothing. ## What a filter looks like A filter criterion is a JSON **event pattern** — the same matching language EventBridge rules use. The pattern mirrors the shape of the record and lists acceptable values: ```json { "body": { "eventType": ["order_placed", "order_cancelled"], "amount": [{ "numeric": [">", 100] }] } } ``` A field present in the pattern must match; a field absent from the pattern is ignored. Beyond exact values, the syntax supports `prefix`, `suffix`, `anything-but`, `numeric` comparisons, `exists`, and `cidr`. Several patterns can be attached to one mapping and they are combined with OR — a record kept by any pattern is kept. In the mapping's API shape, the patterns are strings: ```json { "FilterCriteria": { "Filters": [ { "Pattern": "{\"body\":{\"eventType\":[\"order_placed\"]}}" } ] } } ``` That double encoding — a JSON document inside a JSON string — is the single most common source of a filter that "does nothing". ## The scope you filter on differs by source This is where candidates trip. You filter the **record as the poller sees it**, not the payload as your handler parses it. - **SQS** — the message body arrives as a string in `body`. Lambda will parse it as JSON for you *for matching purposes* when your pattern is scoped to `body`. You can also match on `messageAttributes`. If the body is not valid JSON — plain text, XML, a raw ID — a body-scoped pattern cannot match it, and everything is filtered out. - **DynamoDB Streams** — you filter under `dynamodb`, and the values carry DynamoDB's typed attribute shape (`{"S": "..."}`, `{"N": "..."}`), so patterns must be written against those wrappers. You can also match on `eventName` to keep only `INSERT` or `REMOVE`. - **Kinesis** — the payload is base64-encoded in `kinesis.data`; Lambda decodes it and matches if it is JSON, and you can also match on partition key metadata. ## The consume-not-peek rule This is the part with operational consequences. A filtered-out SQS message is **deleted from the queue** by Lambda, exactly as if you had processed it successfully. It does not stay for another consumer, it does not age into a dead-letter queue, and no metric reports it as a failure — it is simply gone. So a filter is safe only when this Lambda is the *sole* consumer that needs those records, or when whatever else needs them reads from a different subscription. If two consumers need overlapping subsets, fan out first — SNS or an EventBridge bus with two targets, each queue filtered for its own consumer — rather than filtering one shared queue two different ways. For streams the picture is gentler: records are not owned by a consumer, so filtering only advances this mapping's iterator, and other consumers of the same stream are unaffected. ## What filtering does and does not save It eliminates the Lambda invocation, its duration, and its share of concurrency. It does **not** eliminate the source's own charges: SQS still bills the requests the poller makes to receive and delete those messages, and Kinesis still bills the shard. So filtering is an excellent lever on Lambda cost and concurrency and a modest one on messaging cost. ## Failure mode to rehearse A filter that matches nothing does not error. The function goes quiet, the queue drains, `NumberOfMessagesDeleted` stays high, and there is no invocation to inspect. When a filtered function stops firing, verify in this order: is the pattern valid JSON inside a correctly escaped string; is the scope right (`body` versus a top-level field); is the body actually JSON; and does a sample record genuinely satisfy the pattern. Diagnosing this from function logs alone is impossible, because no function ever ran.
- Your filter is deployed, the queue is draining, and the function is never invoked. What do you check?Assume the pattern matches nothing, because that failure is silent — Lambda deletes non-matching SQS messages without logging anything. Check the escaping first, since each pattern is a JSON document embedded in a JSON string. Then check the scope: SQS patterns usually need to be nested under `body`. Then confirm the message body is really JSON, and test the pattern against a captured sample record.
- Two different consumers need overlapping subsets of one queue. Can you solve that with filter criteria?No — filtering on an SQS event source mapping deletes what it rejects, so the second consumer would lose records the first filtered away. Fan out instead: publish to SNS or an EventBridge bus with a queue per consumer, and filter each queue for its own subset. Filtering is safe only where the mapping is the sole consumer of that queue.
- Does filtering reduce your SQS bill as well as your Lambda bill?Only marginally. The poller still calls ReceiveMessage and DeleteMessage for the records it rejects, so SQS request charges remain. What disappears is the Lambda side — invocation count, duration, cold starts and the concurrency those invocations would have held, which is usually the larger number and always the scarcer resource.
saying these in an interview costs you the question
- Believes filtered-out SQS messages stay in the queue for someone else
- Writes the pattern against the payload the handler parses, not the record
- Forgets each pattern is a JSON string that must be escaped
- Expects a non-matching filter to raise an error somewhere
- Assumes filtering removes the source's own request charges