Why is the bronze layer of a lakehouse kept immutable and append-only?

## What "immutable bronze" means in practice An immutable, append-only bronze layer means loads only ever **add** rows. You never update a bronze row to fix a value, never delete rows that look wrong, and never overwrite yesterday's landing table with a corrected extract. Each row carries provenance columns — an ingestion timestamp, a batch or file identifier, sometimes a source offset or row number — so it is always answerable when a record arrived and in which delivery. Corrections from the source arrive as *new* rows, and it is downstream logic, not an in-place edit, that decides which version wins. ## Reason one: downstream tables become recomputable The layering only pays off if silver and gold can be thrown away and rebuilt. That property holds exactly when the raw input is still there and unchanged. A currency conversion applied with the wrong rule, a deduplication that picked the wrong tie-break, a field you parsed as local time when it was UTC — all of these are recoverable by rerunning the transform over retained bronze. If bronze was cleaned on ingest or overwritten each night, the bug is baked into the only copy you have, and "fixing" it means going back to the source system's owner with an apology and a data request. ## Reason two: sources are usually not re-readable This is the reason candidates forget, and it is the strongest one. API endpoints expose rolling windows and then stop serving old data. Event streams have retention measured in days. Change-data-capture feeds emit each change once. Vendors rotate or purge extract files. Operational databases are mutable by definition — reading them again next month gives you *today's* state, not the state you loaded from. The moment you drop raw, you have made the past unrecoverable, and no amount of downstream design compensates. ## Reason three: blame isolation and audit When a finance number is wrong, the first question is whether the source sent bad data or the pipeline mangled good data. With an unmodified bronze row plus its arrival timestamp you answer that in a query rather than an argument. The same property serves audit and regulatory review: you can demonstrate what was received and when, independently of what your transformations concluded. Some teams treat bronze as the system of record for exactly this reason. ## Reason four: schema drift and "we did not need that column yet" Sources add fields. If bronze stores only the columns your current model uses, a field that becomes important next quarter has no history — you can model it going forward but not backwards. Landing the full payload, often as a single semi-structured column plus a few extracted keys, means new requirements can be served retroactively. This is the cheapest form of optionality a platform has. ## What you *do* add on the way in Immutability is not the same as adding nothing. Ingestion metadata is the accepted addition: ```sql CREATE TABLE bronze_orders_raw ( ingest_batch_id VARCHAR(64) NOT NULL, source_file VARCHAR(512) NOT NULL, ingested_at TIMESTAMP NOT NULL, payload VARCHAR(65535) NOT NULL ); ``` That is enough to reconstruct which load produced which downstream row, to reprocess a single batch, and to detect a re-delivery of the same file. ## Duplicates are expected, not a defect Because loaders retry and sources re-send, bronze legitimately contains duplicates. Deduplicating on the way in destroys the record of what actually arrived and hides delivery problems. The duplicates are resolved downstream, where the grain is declared and the tie-break rule is written once and tested. ## The exceptions Two things do modify bronze, and both are deliberate. **Deletion requests** under privacy regulation must be honoured, which means bronze has to be designed so a subject's rows can be located and purged — hard deletes, audited, not routine edits. **Retention expiry** removes data past a defined age, which is a cost and compliance decision made with eyes open: the day you set a 90-day bronze retention, you have decided that reprocessing older than 90 days is impossible. Say that out loud when you set it. ## Cost, and the honest trade-off Keeping everything forever is not free. The usual answers are cheap object storage, retention tiers by dataset value, and compressing or archiving cold partitions — plus being selective about landing genuinely enormous, low-value telemetry. What you should not do is trade away immutability for a modest storage saving; the failure it protects against — an undetectable bug in the only copy of the truth — is far more expensive than the bytes.