skip to content

In Amazon CloudWatch Logs, what is the relationship between a log group and a log stream, and at which of the two do you configure retention? What happens to data already stored when you change that setting?

level: middleimportance: must knowfreq 72%

answer

  1. one is the container, one is a source
  2. configuration lives on the container
  3. the default is forever
  4. the enum, not any integer
  5. lowering it reaches backwards

basics

~20 s

A log group is the container and the unit of configuration; a log stream is one ordered sequence of events from one source inside it. Retention is set per log group, defaults to never expire, and applies retroactively — older events are deleted.

solid answer

~50 s

A log group is the logical bucket for one application or service, and every stream inside it is one ordered sequence of events from a single source — one instance, one Lambda execution environment, one container. Streams are created and abandoned constantly and carry almost no configuration; the log group is where everything you actually manage lives: retention, KMS encryption, tags, metric filters, subscription filters and resource policies. Retention is therefore a log-group property, set with `PutRetentionPolicy`, and it accepts only a fixed list of values such as 1, 7, 30, 90 or 365 days — not any integer. The default is **Never expire**, which is why untouched accounts accumulate years of logs. Changing it is retroactive: events already stored that are older than the new period are deleted automatically, asynchronously rather than instantly, so do not expect the console to shrink the moment you save.

code

bash · 9 lines
bash
aws logs create-log-group --log-group-name /myapp/api

# retention-in-days accepts only an allowed set of values
aws logs put-retention-policy \
  --log-group-name /myapp/api \
  --retention-in-days 30

# back to "Never expire"
aws logs delete-retention-policy --log-group-name /myapp/api

go deeper

for a junior

Recall the two levels and which is which: a log group is the named container, a log stream is one source's events inside it. Say that retention is configured on the group.

for a middle

Explain the mechanics: PutRetentionPolicy takes only allowed values, the default is never expire, and lowering retention deletes already-stored events in the background rather than instantly.

for a senior

Demonstrate operational judgment — enforcing a retention default across an account, knowing that retention only shrinks the storage half of the bill, and moving long-term data to S3 rather than paying CloudWatch storage for years.

for a principal

Own the policy question: what the organisation's default retention is, how it is applied to groups that services create implicitly, and where the durable, legally relevant copy of the logs lives given a log group can be deleted in one call.

## Two levels, and only one of them is configurable CloudWatch Logs has a deliberately shallow hierarchy: - **Log group** — the named container, conventionally path-like: `/aws/lambda/checkout`, `/myapp/api`. This is the resource. It has an ARN, it takes tags, it is what IAM policies and resource policies name, and it holds every setting that matters. - **Log stream** — a sequence of log events **from a single source**, in ingestion order, inside one group. An EC2 instance running the agent gets a stream per instance; ECS gets one per task/container; Lambda gets one per execution environment, with names like `2026/08/21/[$LATEST]a1b2c3…`. The key consequence: streams are ephemeral bookkeeping. They are created automatically, there may be tens of thousands of them, and you cannot attach policy to them. Anything you configure — retention, encryption, filters, alarms fed from filters — is attached to the **group**. ## Retention Set it with the API/CLI: ```bash aws logs put-retention-policy \ --log-group-name /myapp/api \ --retention-in-days 30 ``` Three facts worth knowing precisely: 1. **The default is never expire.** A log group created implicitly — by Lambda on first invocation, by the agent calling `CreateLogGroup` — keeps data forever until someone changes it. This is the single largest source of surprise CloudWatch storage bills, and it is why organisations set retention centrally at group-creation time rather than trusting each team. 2. **`retentionInDays` is an enumeration, not a free integer.** Allowed values include 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827 and 3653 (ten years). Ask for 45 and the API rejects it. `DeleteRetentionPolicy` puts a group back to never-expire. 3. **It is retroactive and asynchronous.** Lowering retention deletes events already stored whose timestamps are older than the new window — it does not merely apply to future data. Deletion is a background process, so storage and the console can lag behind the change; do not conclude the setting did not take because bytes are still visible. There is no per-stream retention and no "archive after N days, delete after M" tiering inside CloudWatch Logs. If you need long-term retention cheaply, you get the data out — an export to S3 or a subscription filter to a delivery stream — and let S3 lifecycle rules handle the tiering. Deleting a log group deletes all of its streams and their events, immediately and irrecoverably. ## Why the group/stream split exists at all Ordering. Within a stream, events are stored and returned in timestamp order, which lets you read one source's history coherently — `GetLogEvents` operates on a single stream. Across streams, you interleave at query time. So the split is what lets CloudWatch Logs be simultaneously a per-source tail and a whole-service search surface. This also explains a design rule: **put one logical application per log group, and let sources be streams.** People who create a group per instance end up unable to configure or query anything coherently — retention, filters and alarms all have to be repeated per group, and cross-instance search means fanning out over hundreds of groups. ## What else lives on the group Because the group is the resource, these are all group-scoped: the KMS key association, the log class (chosen at creation), metric filters, subscription filters, data-protection policies, tags for cost allocation, and resource policies that let another service or account write into it. Every one of those is a reason not to explode your group count. ## Cost shape, briefly You are charged for **ingestion** per GB and for **stored** data per GB-month, and ingestion dominates for most workloads because storage is billed on compressed bytes at a much lower rate. Retention therefore shrinks the smaller half of the bill; it does not touch what you already paid to ingest. That distinction is the one candidates most often miss when asked how to reduce a CloudWatch Logs bill. ## The interviewer's checkpoint Say clearly: group is the configurable resource, stream is one source's ordered sequence, retention is on the group, default is forever, and lowering it deletes existing data. That is the whole answer, and it separates people who have operated CloudWatch Logs from people who have only read about it.

  • Why do Lambda log groups so often end up with never-expire retention?
    Because Lambda creates the group implicitly on first invocation, and an implicitly created group inherits the service default of never expire. Nobody makes a decision, so no retention is set. The usual fix is to create the group explicitly with retention as part of deployment, or to run an account-level sweep that applies a default to any group without a policy.
  • If retention only shrinks the storage half of the bill, how do you attack the ingestion half?
    By ingesting less. Cut log volume at the source, drop debug levels in production, route very high-volume vended logs such as VPC Flow Logs to S3 instead, and consider the Infrequent Access log class for groups you keep for compliance but rarely read. Retention cannot refund bytes you already paid to ingest.
  • How would you keep logs for seven years without paying CloudWatch Logs storage for seven years?
    Get them out. Either export the group to S3, or attach a subscription filter that streams events continuously to Amazon Data Firehose with an S3 destination, then keep a short retention on the log group itself. S3 lifecycle rules move the objects to Glacier tiers, which is dramatically cheaper than long CloudWatch retention.
  • What happens to log streams when you delete a log group?
    They go with it — every stream and every event inside the group is deleted, and there is no undo or recycle bin. That is why long-retention data belongs in S3 with versioning or Object Lock if it has compliance value, rather than living only inside a log group an operator can remove with one API call.

saying these in an interview costs you the question

  • Thinking retention can be set per log stream
  • Assuming CloudWatch Logs defaults to some short retention
  • Believing a shorter retention applies only to future events
  • Expecting storage to drop the instant retention is lowered
  • Creating one log group per instance instead of per application

context

resolving links…