skip to content

An Amazon S3 replication rule targets a bucket in another AWS account. Unencrypted objects replicate fine, but every object encrypted with SSE-KMS shows a replication status of FAILED. What are the likely causes, and what does a correct configuration look like?

level: seniorimportance: should knowfreq 42%

answer

  1. what works narrows what is broken
  2. encrypted objects are opt-in per rule
  3. KMS keys do not cross Regions
  4. two policies for one cross-account key
  5. fix the rule, then re-drive the failures

basics

~20 s

SSE-KMS objects are excluded unless the rule opts them in via SourceSelectionCriteria, and the replication role needs kms:Decrypt on the source key plus kms:Encrypt on a destination-Region key named in the rule. Cross-account also needs both key policies and the destination bucket policy to allow that role.

solid answer

~50 s

The split between unencrypted objects succeeding and SSE-KMS objects failing points squarely at the KMS path, since the destination bucket policy and the role's `s3:ReplicateObject` permission are evidently already correct. Three things are usually missing. First, the rule must opt in to encrypted objects with `SourceSelectionCriteria` setting `SseKmsEncryptedObjects` to Enabled — otherwise they are simply skipped. Second, the destination configuration must name a `ReplicaKmsKeyID`, and because **KMS keys are Regional**, a cross-Region destination needs a key in the destination Region rather than the source key. Third, the replication role needs `kms:Decrypt` on the source key and `kms:Encrypt` on the destination key, **and** both key policies must allow that role — in another account, an IAM policy alone is never sufficient. For cross-account replication I would also confirm ownership handling, since replicas otherwise remain owned by the source account.

code

json · 23 lines
json
{
  "Role": "arn:aws:iam::111122223333:role/s3-replication-role",
  "Rules": [
    {
      "ID": "kms-cross-account",
      "Priority": 1,
      "Status": "Enabled",
      "Filter": {},
      "DeleteMarkerReplication": { "Status": "Disabled" },
      "SourceSelectionCriteria": {
        "SseKmsEncryptedObjects": { "Status": "Enabled" }
      },
      "Destination": {
        "Bucket": "arn:aws:s3:::example-dest-bucket",
        "Account": "444455556666",
        "AccessControlTranslation": { "Owner": "Destination" },
        "EncryptionConfiguration": {
          "ReplicaKmsKeyID": "arn:aws:kms:eu-west-1:444455556666:key/1234abcd-12ab-34cd-56ef-1234567890ab"
        }
      }
    }
  ]
}

go deeper

for a junior

Know that objects encrypted with SSE-KMS need the replication rule to opt them in explicitly, and that the replication role needs permission on the KMS keys as well as the buckets.

for a middle

Explain the moving parts: SourceSelectionCriteria for encrypted objects, ReplicaKmsKeyID naming a key in the destination Region, and kms:Decrypt on the source key with kms:Encrypt on the destination key.

for a senior

Debug it in order — use what still works to narrow the fault, read the destination key policy from the destination account, then re-drive failed objects with a batch job rather than assuming the fix is retroactive.

for a principal

Own the cross-account contract: who provisions the destination key and bucket policy, how those grants are reviewed, and how you detect silently failing replication before a failover discovers the gap for you.

## Read the symptom first The useful signal here is what *does* work. Unencrypted objects replicate, which proves that versioning is on at both ends, that the replication role is assumable by S3, that the destination bucket policy grants that role `s3:ReplicateObject`, and that the rule's filter matches. Everything structural is fine. The failure is confined to the encryption path, and that narrows the investigation to three specific pieces of configuration. ## Cause 1: the rule never opted encrypted objects in By default a replication rule does not replicate objects encrypted with SSE-KMS. You enable them explicitly: ```json "SourceSelectionCriteria": { "SseKmsEncryptedObjects": { "Status": "Enabled" } } ``` Note the diagnostic difference: objects that were never opted in are **skipped** rather than reported as failures, so if you see genuine `FAILED` statuses, the opt-in is probably already there and one of the next two causes applies. Both are worth checking in order anyway, because the fix is cheap and the confusion is common. ## Cause 2: no usable destination key The rule's destination configuration must name a KMS key for the replica: ```json "EncryptionConfiguration": { "ReplicaKmsKeyID": "arn:aws:kms:eu-west-1:444455556666:key/1234abcd-..." } ``` **KMS keys are Regional resources.** A key in `us-east-1` cannot encrypt an object being written into a bucket in `eu-west-1`. This trips people who copy the source key ARN into the field and cannot understand why nothing works — the ARN is valid, it is simply in the wrong Region. Multi-Region keys exist as a distinct KMS feature; a normal key does not travel. ## Cause 3: the role can read but not re-encrypt Replication of an encrypted object is a decrypt-then-encrypt operation performed by S3 using the replication role's identity. The role therefore needs: - `kms:Decrypt` on the **source** key, so the object can be read; - `kms:Encrypt` on the **destination** key, so the replica can be written. AWS's role examples scope these with the encryption-context condition key `kms:EncryptionContext:aws:s3:arn`, which limits the grant to objects from a specific bucket. That scoping is good practice and also a common source of a self-inflicted denial when the ARN in the condition does not match the bucket in play. ## The cross-account multiplier Everything above is doubled by the account boundary. For a role in account A to use a KMS key in account B, **two** policies must allow it: the role's IAM policy *and* the key policy on the destination key. This is the standard KMS cross-account model, and it is where most of these incidents actually live — a platform team creates the destination key with a default key policy that names only its own account, and no IAM policy in the source account can overcome that. The same doubling applies to the bucket: the destination bucket policy must grant the source account's replication role `s3:ReplicateObject`, `s3:ReplicateTags` and, when delete markers replicate, `s3:ReplicateDelete`. ## Ownership: the quiet second problem Cross-account replicas are, by default, owned by the **source** account, which means the destination account owns the bucket but not the objects inside it and can be unable to read its own data. The rule fixes this with an ownership override — `AccessControlTranslation` with `Owner` set to `Destination` — which additionally requires the replication role to hold `s3:ObjectOwnerOverrideToBucketOwner`. Raising this unprompted is a strong signal in an interview, because it is the bug that surfaces a week after replication starts working. ## How to debug it methodically 1. Confirm the failure is real: `aws s3api head-object --bucket <source> --key <key> --query ReplicationStatus`. `FAILED` means S3 tried and was denied; no status at all means the object was never eligible. 2. Read the rule JSON and check `SourceSelectionCriteria` and `ReplicaKmsKeyID` — including the **Region** in that key ARN. 3. Check the replication role for `kms:Decrypt` on the source key and `kms:Encrypt` on the destination key, and check the encryption-context conditions. 4. Read the **destination key policy** from the destination account and confirm it names the source account's replication role. 5. Confirm ownership override if the destination account needs to read the replicas. 6. Once fixed, run **S3 Batch Replication** to re-drive the objects that failed while the configuration was wrong — a corrected rule does not retroactively retry them. That last step is what separates a complete answer from a partial one: fixing the configuration stops new failures, but the backlog of already-failed objects stays failed until you replay it.

  • Why is an IAM policy on the replication role not enough to use the destination account's KMS key?
    Cross-account KMS access requires both sides to agree: the key policy in the owning account must allow the external principal, and the caller's IAM policy must allow the action. Neither alone suffices. That is why these incidents usually resolve in the destination account, where a default key policy naming only its own account silently blocks every replication attempt.
  • After fixing the configuration, do the previously failed objects replicate on their own?
    No. A corrected rule applies to new writes; objects already marked FAILED stay failed. You clear the backlog with an S3 Batch Replication job, which can target objects that previously failed and replays them through the now-correct replication configuration. Skipping this step leaves a silent hole in the replica that only surfaces during a failover.
  • What symptom tells you the ownership override is missing on a cross-account rule?
    Objects arrive in the destination bucket, but the destination account gets access-denied reading them, because replicas default to source-account ownership. The fix is AccessControlTranslation with Owner set to Destination on the rule, plus s3:ObjectOwnerOverrideToBucketOwner on the replication role. It typically surfaces a week after replication starts working, when someone finally reads the replica.
  • How do you tell an object that failed replication from one that was never eligible?
    Check the object's replication status with HeadObject. FAILED means S3 attempted the copy and was denied — a permissions or key-policy problem. No replication status at all means the object was never in scope, because it predates the rule, falls outside the filter, or was an SSE-KMS object on a rule that never opted encrypted objects in.

saying these in an interview costs you the question

  • Assuming SSE-KMS objects replicate without opting in
  • Reusing the source KMS key ARN for another Region
  • Granting the role KMS access without editing the key policy
  • Expecting failed objects to retry automatically after a fix
  • Ignoring that cross-account replicas stay owned by the source account

context