Google Cloud Storage (GCS) is Google Cloud's object storage service - durable, scalable buckets that hold files of any size. In a data stack it is the lake layer: where raw and curated files live before and after processing, and the storage that BigQuery, Dataproc (Spark/Hadoop), and Dataflow read from and write to.

Buckets, objects, and the flat namespace

Data lives in buckets as objects keyed by name. Like S3, the namespace is flat - slashes in names render as folders in the console but there are no real directories. Objects are immutable: you replace, not edit. Buckets are created in a location (regional for lowest latency to co-located compute, multi-region for availability), and you keep analytics buckets in the same region as the engines that read them to avoid egress cost and latency.

Storage classes

GCS tiers cost against access frequency: Standard for hot data, Nearline (≥30-day), Coldline (≥90-day), and Archive (≥365-day) for progressively colder, cheaper storage. Object Lifecycle Management rules move or delete objects automatically by age - for example, transition raw landing data to Nearline after 30 days and delete after a year. Because analytic data is mostly write-once and read-rarely-after-a-while, lifecycle rules meaningfully cut the bill.

Access from engines

Engines address GCS with the gs:// scheme and authenticate via service accounts (or workload identity on GKE), governed by IAM at the bucket or project level.

# Spark on Dataproc reading Parquet from GCS
df = spark.read.parquet("gs://lake/events/")

# BigQuery external table over lake files — query without loading
CREATE EXTERNAL TABLE dataset.events
OPTIONS (format = 'PARQUET', uris = ['gs://lake/events/*.parquet']);

The BigQuery relationship

GCS is the lake half of a common Google pattern: land and stage data in GCS, then either load it into BigQuery's managed storage for fastest queries, or leave it in GCS and query via BigQuery external tables / BigLake for a lakehouse approach that avoids duplication. BigLake adds fine-grained security and Iceberg support over GCS files, narrowing the gap between "in the warehouse" and "on the lake."

Organizing a lake on GCS

  • Separate buckets or prefixes per zone (raw / curated / published) with distinct IAM and lifecycle policies.
  • Store curated data as Parquet, and as Iceberg/Delta when you need transactions.
  • Keep buckets and compute in the same region; watch egress for cross-region or internet reads.
  • Enable uniform bucket-level IAM rather than legacy per-object ACLs for manageable security.

Where it fits

GCS is the lake storage when your stack is on Google Cloud - the counterpart to Amazon S3 and Azure ADLS. The analytic patterns are identical across all three clouds; the choice follows your platform and, secondarily, where your compute and the rest of your data already live.

Category note

GCS is object storage, not a file format - it's where your Parquet/Iceberg/Delta files live, grouped here as part of the storage layer.