A data retention policy defines how long each kind of data is kept and what happens when that window expires - archive it, anonymize it, or delete it. Without one, warehouses accumulate everything forever, which quietly raises storage cost, expands the surface area of a breach, and creates legal exposure by holding data longer than regulations permit. A good retention policy is the deliberate answer to "how long should we keep this, and why?"

What drives the window

Retention periods come from three pressures that often pull in opposite directions. Regulation sets both floors and ceilings - financial records may need to be kept for seven years under SOX, while GDPR's data-minimization principle says personal data shouldn't be kept longer than necessary for its purpose. Business need wants history for trend analysis and machine-learning training. Cost and risk push the other way: every retained row costs storage and is one more record exposed if the warehouse is breached. The policy resolves these per data category rather than with a single blanket rule.

Tiers, not just delete

Expiration rarely means immediate deletion. Mature retention works in tiers: hot data stays in fast, queryable storage; older data moves to cheaper cold or archival storage (S3 Glacier, for example) where it's rarely read but still recoverable; and only past the legal limit is it permanently deleted. Anonymization is a middle path - strip or hash the identifying columns and keep the rest for analytics, which can take data out of scope for privacy regulations while preserving its analytical value. The warehouse usually expresses these with partition lifecycle rules, TTL settings, and storage-tiering policies.

Enforcing it as code

A retention policy that lives in a wiki gets ignored; one enforced by automation actually holds. The pattern is to attach a retention period to each data classification, then run scheduled jobs that archive, anonymize, or drop partitions past their window - and record what they did for audit. This connects directly to compliance: the ability to honor a GDPR erasure request or prove you didn't keep payment data too long depends on retention being a real, executed process, not a stated intention. It also depends on knowing where the data is, which is where lineage comes in.

Frequently asked questions

Why not just keep everything? Indefinite retention raises storage cost, enlarges breach exposure, and can violate regulations that forbid keeping personal data longer than necessary.

Does expiration always mean deletion? No - mature policies tier data from hot to cold to archival storage, anonymize where possible, and only delete past the legal limit.

What is anonymization's role? Stripping or hashing identifying columns keeps data useful for analytics while often removing it from the scope of privacy regulations.

How are retention policies enforced? By attaching a retention window to each data classification and running scheduled jobs that archive, anonymize, or drop expired partitions, with an audit record of the action.