Encryption in Transit Versus Encryption at Rest

Encryption in transit and encryption at rest protect data in two different states, and Google Cloud encrypts both states by default. Encryption in transit protects data while it moves across networks, while encryption at rest protects data while it is stored on disks or other persistent storage. The difference matters because moving data can be intercepted on the network, while stored data can be exposed if a disk, backup, or snapshot is accessed without permission.

Dataflow (opens in a new tab) pipelines show both states at once because data flows between workers and services during processing and also rests on worker disks and snapshots. Dataflow runs on managed, autoscaling virtual machines called workers, so pipeline settings decide how well data stays protected. Securing a pipeline involves three main levels: network and infrastructure security, Identity and Access Management, and data protection and encryption. Network controls mainly strengthen transit protection, while key controls add extra assurance for stored data.

1.1Transit Protection While Data Moves Between Services

Transit protection is the default encryption applied to data traveling between systems, such as from Dataflow workers to Google APIs like Cloud Storage, BigQuery, or Pub/Sub. Transit protection keeps moving data unreadable so interception does not expose sensitive pipeline content. By default, Dataflow workers receive public IP addresses to reach the internet and Google APIs, which increases network exposure even though transit encryption stays active.

Traffic flows from workers to Google services, and administrators can control the path that traffic takes:

• Disable public IP addresses on workers by passing the parameter --usePublicIps=false during pipeline submission.
• Enable Private Google Access on the Virtual Private Cloud (VPC) subnet so workers without public IPs can still route traffic to Google services internally.
• Place pipelines inside a security perimeter using VPC Service Controls (VPC-SC) and use dedicated subnets with strict firewall rules to block unwanted traffic, including SSH.

A VPC Service Controls perimeter creates a trust boundary around supported resources. Data and authorized services stay inside the perimeter on one side, while external resources sit outside the perimeter on the other side, so data cannot be copied outside even by users with valid credentials. Choose this combination when a pipeline moves sensitive data between workers, storage, and analytics services and must limit network paths and exfiltration risk.

1.2At-Rest Protection While Data Sits on Disks and Snapshots

At-rest protection is the default encryption applied to data stored on persistent resources, such as worker boot disks, data disks, snapshots, and machine images. At-rest protection keeps stored data unreadable if a disk is lost, reused, or accessed without authorization. While Google encrypts stored data by default, highly regulated workloads often need extra customer control over the encryption keys.

Three key management options provide different levels of customer control for Compute Engine storage used by workloads such as Dataflow:

• Google-owned and managed keys are created and rotated by Google without customer action.
• Customer-managed encryption keys (CMEKs) are created and managed by the customer in Cloud Key Management Service (Cloud KMS) (opens in a new tab), including optional automatic rotation and on-demand creation through Cloud KMS Autokey.
• Customer-supplied encryption keys (CSEKs) are supplied directly by the customer for each operation.

1.3Network and Key Controls That Strengthen Each State

Network and key controls build on top of default encryption, and the choice of storage determines the choice of key control. Compute Engine rotates the Google-owned and managed keys used to protect data on a yearly basis, which limits the impact of a compromised key. If you use CMEKs, enable automatic rotation for continued protection.

The table shows that Persistent Disk offers the most key choices, while Hyperdisk and Local SSD are more limited. Protection flows from a source disk to its clones and snapshots, so the key rules for the source affect its copies.

Core elements of encryption are the separate protections for moving data and stored data within Google Cloud. Encryption in transit protects data while it moves between systems, and encryption at rest protects data while it is stored on disks and snapshots. Google Cloud encrypts all data in transit and at rest by default, with options for customers to control the keys used for stored data.

2.1Principles of Transit Encryption Between Sources and Workers

Transit encryption is the protection applied to data during transmission between services and systems. In Google Cloud, data flows from sources such as Cloud Storage, BigQuery, or Pub/Sub to processing workers and then to destination systems, and that moving data is encrypted by default. The principles of transit encryption include the use of secure transport protocols such as TLS/SSL to prevent interception while data travels over the network.

Dataflow (opens in a new tab) workers run on managed virtual machines that by default use public IP addresses to reach Google APIs. Traffic can instead stay inside the private network by disabling public IPs with --usePublicIps=false and enabling Private Google Access on the VPC subnet, so workers route traffic to Google services internally. Placing pipelines inside a VPC Service Controls perimeter further prevents data from being copied to resources outside the perimeter. The order is fixed: first the worker sends traffic, then the private network path carries it to Google services, and only after the perimeter check does it reach its destination.

2.2Principles of Disk-Level Encryption for Stored Copies

Disk-level encryption is the protection applied to data stored on persistent media, such as virtual-machine disks, clones, machine images, and snapshots. Disk-level encryption secures the stored blocks so the data remains unreadable without access to the encryption key. This protection applies even when disks are cloned, imaged, or backed up as snapshots, so a copy inherits the need for key-based access.

The table shows that choice narrows for newer and ephemeral storage. Hyperdisk volumes cannot use CSEKs, and Local SSD disks support only Google-owned and managed keys. Snapshots must use the same key as the source disk, so protection flows from the source disk to its snapshot copy.

2.3Key Control and Rotation for Stored Data

Key control for stored data defines who manages the encryption keys that protect data at rest. Google-owned and managed keys require no customer action and are rotated by Compute Engine on a yearly basis. Customer-managed encryption keys (CMEK) are created and managed in Cloud Key Management Service (Cloud KMS) (opens in a new tab), and Google recommends enabling automatic rotation for CMEKs.

Cloud KMS Autokey simplifies CMEK use by generating key rings and keys on demand as part of resource creation in Compute Engine. Autokey also creates the service agents that perform encrypt and decrypt operations and grants them the required Identity and Access Management (IAM) roles. Snapshots protected by Autokey do not get new keys and must use the same key as the source disk. Customer-supplied encryption keys (CSEK) are deprecated for Compute Engine, so new designs should use Google-owned keys or CMEKs.

Technical protocols for encryption depend on the state of the data. Encryption in transit protects data while it moves across networks, and encryption at rest protects data while it is stored on disks and snapshots. Google Cloud encrypts all data in transit and at rest by default, with added controls available for highly regulated needs.

3.1How TLS Protects Moving Data From Interception

Encryption in transit is protection applied to data while it travels between services, such as from a pipeline worker to Cloud Storage, BigQuery, or Pub/Sub. This state uses Transport Layer Security (TLS) (opens in a new tab), which scrambles data during transmission so intercepted traffic cannot be read. TLS therefore mitigates network interception threats like eavesdropping on public or shared networks.

Network controls in Dataflow reduce exposure during transit. Dataflow workers run on managed virtual machines that are assigned public IP addresses by default to reach the internet and Google APIs. For a secure environment, traffic flows from the workers to Google services through internal paths instead of the public internet:

• Disable public IPs on Dataflow workers with --usePublicIps=false so workers have no direct internet address.
• Enable Private Google Access on the VPC subnet so workers without public IPs can still route traffic to Google services internally.
• Place pipelines inside a security perimeter with Shared VPCs and VPC Service Controls (VPC-SC) so data cannot be copied to resources outside the perimeter, even by users with valid credentials.

The perimeter acts as a trust boundary with protected data inside and outside resources on the other side. The overall flow is ordered: first workers generate traffic, then the private subnet routes it internally, and only then do Google services receive it if perimeter policy allows.

3.2How AES Protects Stored Data From Physical Access

Encryption at rest is protection applied to data stored on media, such as virtual-machine disks, clones, machine images, and snapshots. This state uses the Advanced Encryption Standard (AES), which scrambles stored blocks so stolen or discarded media cannot be read without the key. AES therefore mitigates unauthorized physical access to storage media and improper reuse of disks.

Compute Engine (opens in a new tab) disk encryption depends on the disk type and the key type. The key types are Google-owned and managed keys, customer-managed encryption keys (CMEK) managed in Cloud Key Management Service (Cloud KMS), and customer-supplied encryption keys (CSEK):

The table shows that choice narrows for newer and ephemeral storage. Hyperdisk volumes cannot use CSEKs, and Local SSD disks only support Google-owned and managed keys. Snapshots must use the same key as the source disk, so protection flows from the source disk to its snapshot copy.

3.3Ownership and Rotation That Limit Key Compromise

Key ownership determines who controls the AES keys that protect stored data. Google-owned and managed keys require no customer action, while CMEK lets a team create and manage keys in Cloud KMS for added control. CMEKs can be created manually or generated on demand with Cloud KMS Autokey during resource creation, where service agents are created and granted Identity and Access Management (IAM) roles for encrypt and decrypt operations.

Rotation limits the impact of a compromised key by replacing it over time. Compute Engine rotates the Google-owned and managed keys used to protect data on a yearly basis. For CMEKs, Google recommends enabling automatic rotation, so the key version changes on a schedule without manual replacement.

Key Facts

• Google Cloud encrypts data in transit while it moves between workers and services and encrypts data at rest while it sits on disks and snapshots, both by default.
• Transport Layer Security (TLS) protects moving data from network interception, while Advanced Encryption Standard (AES) protects stored data from unauthorized physical access.
• Disabling public IPs with --usePublicIps=false plus Private Google Access lets Dataflow workers route traffic to Google services internally, and a VPC Service Controls perimeter blocks copies to outside resources.
• Persistent Disk volumes support Google-owned keys, customer-managed encryption keys (CMEK), and customer-supplied encryption keys (CSEK), while Hyperdisk volumes do not support CSEKs and Local SSD disks support only Google-owned keys.
• Compute Engine rotates Google-owned keys on a yearly basis, and customer-managed encryption keys should use automatic rotation in Cloud Key Management Service.