Design a hardened consumer-side deserialization strategy for a service that processes events from a partially-trusted set of producers. What controls would you put in place?

## Frame it as a boundary The consumer is a **trust boundary**: bytes arriving from partially-trusted producers are untrusted input. A hardened design applies **defense in depth** so no single failure is catastrophic. ## The hardened design - **1. Choose a safe format (eliminate the worst vector).** Mandate schema-validated, data-only encodings — Avro or Protobuf behind a Schema Registry, or JSON Schema bound to fixed DTOs. Ban native Java serialization (`ObjectInputStream`) and Jackson polymorphic/default typing, because both let the payload choose which classes are instantiated (gadget chains → RCE). This single decision removes the entire code-execution class of attacks. - **2. Allowlist trusted types (don't decode the unexpected).** Even with a safe format, decode only what you expect: pin to known schema subjects/IDs and concrete `SpecificRecord`/generated classes, and reject or dead-letter records whose schema ID or type isn't on the allowlist; if polymorphism is genuinely required, restrict it with a `BasicPolymorphicTypeValidator` allowlist rather than open typing. This blocks schema-spoofing and unexpected-shape attacks. - **3. Authenticate identities at both boundaries.** On the brokers, require producer authentication (SASL/SCRAM or mTLS) and apply topic ACLs so only intended producers can write — 'partially trusted' should still be *authenticated*. On the registry, require Basic or mTLS auth, set `auto.register.schemas=false` on consumers and untrusted producers, and restrict who may register/evolve schemas. This shrinks the write surface and prevents silent schema mutation. - **4. Bound resources (stop DoS).** A small payload can describe a huge or deeply nested structure. Cap maximum message size, recursion/nesting depth, and array/map cardinality; reject oversize records early. This defends against expansion/decompression bombs and memory/stack exhaustion that a 'safe' format alone doesn't prevent. - **5. Fail safe on bad records (poison-pill handling).** A single record that fails to deserialize must not stall or crash the consumer group. In Spring Kafka, wrap the deserializer in `ErrorHandlingDeserializer` and configure a `DefaultErrorHandler` with a `DeadLetterPublishingRecoverer` so the offending record is routed to a dead-letter topic and the consumer advances. Without this, a poison pill is re-fetched forever, halting progress (an availability incident). Decide DLQ retention, alerting, and replay policy up front. - **6. Validate values, not just shape.** Type-safety is not value-safety. A well-formed record can carry a malicious URL, an out-of-range amount, or an oversized string. Apply business-rule validation (bounds, formats, allowlisted enums) after deserialization, and treat downstream uses (SSRF-prone URLs, SQL, template engines) with their own input handling. - **7. Observe and respond.** Emit metrics/alerts for deserialization failures, DLQ volume, and records carrying unexpected schema IDs or types — these are both reliability and security signals. A spike can indicate a compromised producer or an attempted spoofing campaign. ## Tradeoffs a principal weighs - Strict allowlists and `auto.register.schemas=false` add operational friction when legitimately evolving schemas — mitigate with a governed registration workflow. - DLQs need ownership, retention, and replay tooling or they become silent data-loss sinks. - mTLS everywhere is the strongest identity story but adds certificate-lifecycle burden; Basic auth is simpler but introduces shared secrets. The art is matching control strength to the trust level of each producer set while keeping evolution practical.