Regulatory Compliance for Data Architectures on Google Cloud

Establishing cryptographic control and auditable compliance means owning the encryption keys that protect data at rest and proving how those keys are used, accessed, and retained. In Google Cloud, Cloud Key Management Service (Cloud KMS) (opens in a new tab) centralizes key creation, rotation, destruction, and access control, while Customer-Managed Encryption Keys (CMEK) let supported services encrypt data with those customer-controlled keys. Combined with organization policies, key usage tracking, and audit logging, this model supports data sovereignty, regulatory compliance, and crypto-shredding when data must be selectively destroyed.

1.1Data-at-Rest Encryption and Envelope Encryption

Data-at-rest encryption in Google Cloud starts with Google default encryption using FIPS 140 validated AES-256. Transparent encryption also protects traffic across the Google backbone with MACsec, virtual machine-to-virtual machine traffic in Virtual Private Cloud (VPC) networks, application-layer traffic with TLS, and on-premises connections with Cloud VPN tunnels. Envelope encryption is the mechanism that links customer control to that default protection. Each service generates a data encryption key (DEK) to encrypt data, then wraps the DEK with a key encryption key (KEK) managed in Cloud KMS. Services that support CMEK use customer Cloud KMS keys as KEKs to wrap their DEKs, so revoking the KEK removes the ability to unwrap DEKs and read the data. Decrypted key material cannot be exported or viewed through APIs, Google personnel cannot access unencrypted customer key material, and Hardware Security Module (HSM) key material is never accessed in decrypted state by Cloud KMS jobs.

1.2Customer-Managed Encryption Keys with Cloud KMS

Customer-Managed Encryption Keys (CMEK) (opens in a new tab) are Cloud KMS keys that customers own and manage to protect resources in CMEK-integrated services such as BigQuery, Cloud Storage, Compute Engine disks, and Secure Source Manager. Using CMEK gives control over protection level, location, rotation schedule, Identity and Access Management (IAM) permissions, cryptographic boundaries, lifecycle, and audit logs, while accessing the resource remains similar to Google default encryption. Cloud KMS Autokey automates provisioning by generating key rings and keys on demand when a resource is created and by creating service agents and granting required IAM roles if they do not already exist.

Configuring CMEK follows an ordered dependency between the key, the service identity, and the resource:

1. Create key ring and key: Create or choose a Cloud KMS key ring geographically near the resources to protect, in the same project or a separate key-management project, then create or import the key that becomes the CMEK.
2. Grant service access: Grant the roles/cloudkms.cryptoKeyEncrypterDecrypter role on the CMEK to the service agent for the protected service.
3. Attach key at creation: Configure the resource to use the CMEK when the resource is created, in the same location as the key.
4. Operate through the service: Allow requesters to access data through the service without direct access to the CMEK, as long as the service agent retains its role.

Using a separate key-management project supports separation of duties and tighter IAM control. If the service agent role is revoked, or if the CMEK is disabled or destroyed, that data cannot be accessed, and some services can experience permanent data loss if the key remains disabled or inaccessible for too long.

1.3Key Protection Levels and External Key Management

Cloud KMS protection levels define where key material lives and what compliance assurance it provides, and the choice controls availability, rotation options, quota behavior, and FIPS validation. Software keys use FIPS 140-3 Level 1 validated modules and support automatic rotation for symmetric keys, while hardware keys use FIPS 140-2 Level 3 validated HSMs for the highest compliance. Cloud External Key Manager (Cloud EKM) (opens in a new tab) extends CMEK to keys that reside outside Google Cloud in a supported external key manager, connected over the internet or over VPC, but some services that support Cloud KMS keys do not support Cloud EKM keys. The protection options are:

• Software keys: Generate symmetric keys with automatic rotation or asymmetric keys with manual rotation in all Google Cloud locations, with control over rotation period, IAM, and organization policies.
• Imported software keys: Import externally created key versions into Cloud KMS and rotate manually by importing new versions, with IAM and organization policy governance.
• Hardware keys with Multi-tenant Cloud HSM: Generate keys in FIPS 140-2 Level 3 HSM clusters managed by Google Cloud, exposed through Cloud KMS for CMEK integration and rotation control.
• Hardware keys with Single-tenant Cloud HSM: Generate keys in dedicated HSM partitions that customers control, where instance operations require a quorum of designated administrators and two-factor authentication with security keys owned outside Google Cloud, providing cryptographic isolation.
• External keys with Cloud EKM: Use keys held externally to protect data at rest in supported CMEK services or by calling the Cloud KMS API directly, while retaining external availability responsibility.

CMEK-encrypted resources consume Cloud KMS cryptographic request quotas at creation time, and ongoing encrypt and decrypt operations affect quotas only when using hardware or external keys. Some services offer CMEK compliance instead of full CMEK integration, meaning temporary data and ephemeral keys are never written to disk.

1.4Customer-Supplied Encryption Keys Compared with Other Options

Customer-supplied encryption keys (CSEK) are keys where customers store the key material and supply it to Cloud Storage or Compute Engine when needed, and Google Cloud does not store CSEKs in any way. CSEK support is not a feature of Cloud KMS, so CSEKs do not provide centralized rotation, IAM governance, usage tracking, or audit logging through Cloud KMS. CSEKs for Compute Engine are deprecated, with CMEK as the alternative, while Cloud Storage continues to document CSEK alongside client-side encryption where data is encrypted in the customer application before upload. The three server-side models compare as follows:

1.5Enforcing Consistent Use and Proving Compliance

Auditable compliance in Cloud KMS depends on preventing noncompliant encryption, tracking where keys are used, and logging who touched keys and data. Organization policies let administrators require CMEK usage and constrain which Cloud KMS keys qualify for protection, including allowed key projects, protection levels, locations, and controls on key version destruction. Key usage tracking shows the resources, projects, and unique Google Cloud products protected by a specific CMEK and whether keys are in use. CMEK also supports regulatory goals such as owning keys, establishing a single-tenant cryptographic boundary, selectively deleting data by destroying keys, and meeting current or future rules that require customer key control. Administrative and data access to encryption keys can be logged through Cloud KMS audit logging, and Key Access Justifications integrates with Cloud KMS and Cloud EKM to provide a detailed justification each time a key is requested for encrypt or decrypt, with an automated policy to approve or deny access. Together with Access Approval and Access Transparency, Key Access Justifications lets customers deny Google the ability to decrypt data, making the customer the arbiter of access.

1.6Secure Governed Pattern for Sensitive Data

The secure governed pattern for sensitive data combines CMEK, automated classification, and a network perimeter that overrides permissive IAM settings. In this architecture, a user with appropriate IAM permissions uploads data to BigQuery or Cloud Storage configured for CMEK with Cloud KMS Autokey, and the storage service encrypts the data with a CMEK fetched from Cloud KMS. Sensitive Data Protection then continually inspects, classifies, and de-identifies sensitive data such as personally identifiable information (PII) using configured masking templates, while the storage services sit inside a VPC Service Controls perimeter. Access across that perimeter boundary is governed separately from resource IAM: designated IAM principals receive access through an Access Context Manager access level template, and VPC Service Controls denies access to anyone not specified in the template, even if resources are misconfigured. This layered flow — data flows from the uploader to the storage service, key material flows from Cloud KMS Autokey to the storage service, classification flows from Sensitive Data Protection to the stored data, and access decisions flow from Access Context Manager to the perimeter — ensures encryption with customer-controlled keys, automatic detection and masking, and strictly confined contextual access for regulated and unregulated workloads.

Implementing data privacy and de-identification techniques means removing or obscuring personally identifiable information (PII) and protected health information (PHI) while preserving data utility for analytics. Sensitive Data Protection (opens in a new tab) (which includes Cloud Data Loss Prevention (Cloud DLP)) provides inspection and transformation, and BigQuery, Cloud Storage, Dataflow, and Cloud Run provide the batch, streaming, and query-time paths where those transformations are applied. Together they support data minimization, regulatory compliance, and controlled re-identification.

2.1Data Minimization and Classification

Data minimization means not collecting, keeping, or using data that is not strictly necessary for business goals, and using synthetic or fully anonymized data where possible. Sensitive Data Protection classifies confidential data such as credit card numbers, addresses, and phone numbers using more than 200 built-in classifiers called infoTypes. Classification results can be tagged, tokenized before storage, and registered in Knowledge Catalog for lineage, data quality, discovery, and governance. This governance layer monitors data flows, transformations, and access so audit logs can prove who accessed or changed sensitive data.

2.2De-identification Transformations That Preserve Utility

De-identification in Sensitive Data Protection is the removal of identifying information so health, financial, or demographic data can be used and shared under privacy requirements. De-identification preserves utility for joining or analytics by replacing raw identifiers rather than deleting entire records. Supported transformation families include:

• Pseudonymization and tokenization: Replace a sensitive value with a reversible token or surrogate that can be re-identified only by authorized processes.
• Masking and redaction: Partially or fully obscure characters, including redaction of sensitive regions in images.
• Generalization and bucketing: Replace precise values with broader ranges, such as age buckets or generalized locations.
• Cryptographic de-identification: Apply deterministic or format-preserving encryption and hashing so the same input produces the same output for joins without exposing plaintext.

For structured data, Sensitive Data Protection applies record transformations based on fields, for example checking for credit card numbers and card PINs. For unstructured data, it applies infoTypes to find and transform matches in free text.

2.3Batch and Streaming De-identification During Ingestion

Ingestion-time de-identification de-identifies confidential data before it lands in the warehouse, which is required when analysts must not see raw values. Data flows from the source to Cloud Storage or Pub/Sub, then through a Dataflow pipeline that calls Sensitive Data Protection, then to a BigQuery warehouse split into confidential and non-confidential projects. Tokenized data is persisted to BigQuery so downstream batch and streaming consumers read only de-identified values. An alternative application-layer pattern tokenizes data in the ingestion pipeline with Sensitive Data Protection before persisting to BigQuery, so a data scientist can analyze PII-containing datasets without read access to raw sensitive fields. The ingestion pipeline depends on three key groups:

• Ingestion CMEK: A customer-managed encryption key (CMEK) that protects the Dataflow and Pub/Sub ingestion path.
• Wrapped de-identification key: A cryptographic key wrapped by Cloud Key Management Service (Cloud KMS) with Cloud Hardware Security Module (Cloud HSM) that Sensitive Data Protection uses to tokenize or encrypt confidential values.
• Warehouse CMEKs: Separate CMEKs for the BigQuery dataset in the non-confidential data project and the dataset in the confidential data project.

2.4Query-Time De-identification with BigQuery Remote Functions

Query-time de-identification sanitizes results in transit without maintaining a separate de-identified copy or a Dataflow cluster. BigQuery remote functions call a central de-identification service hosted on Cloud Run, so one service can serve de-identification and re-identification across the organization. A query calls functions such as fns.dlp_freetext_encrypt() on a PII column, data flows from BigQuery to the Cloud Run service for transformation, and the encrypted or decrypted result flows back into the query result. This pattern supports policies that store only de-identified data in the warehouse but still allow authorized re-identification for reporting, and it avoids handling large volumes of sensitive data in persistent pipelines.

2.5Storage De-identification for Cloud Storage Files

Storage de-identification creates de-identified copies of files in Cloud Storage while leaving source files unaltered. The operation starts when you create an inspection job (DlpJob) configured for de-identification, then Sensitive Data Protection scans files in the specified input location, de-identifies data matching your inspection criteria, and writes transformed content to new files in a designated output bucket. The output preserves folder hierarchy and can use separate buckets with distinct Identity and Access Management (IAM) controls for compartmentalization. Common uses are anonymized datasets for development and training, sanitized repositories for third parties or auditors, automated privacy pipelines for newly uploaded documents, and isolated mirror copies with restricted access.

2.6Key Management, Column Encryption, and Column Access

Key management and column controls protect data that remains sensitive after de-identification and control who can reverse it. Cloud KMS with Cloud HSM manages CMEKs, the CMEK location must match the protected resources, and rotation defaults to every 30 days. If compliance requires keys outside Google Cloud, Cloud External Key Manager (Cloud EKM) holds external keys, but the customer then owns rotation and availability. For external sources, client-side encryption with the Tink library encrypts columns before upload; Tink is compatible with BigQuery Authenticated Encryption with Associated Data (AEAD) functions for later decryption. Client-side column encryption uses a two-key envelope where a data encryption key (DEK) encrypts the column and a key-encryption key (KEK) wraps the DEK. The DEK can be generated locally or from Cloud HSM, while the KEK is a symmetric CMEK keyset stored in Cloud HSM and managed with IAM. During ingestion both the encrypted data and the wrapped DEK are stored in BigQuery in separate tables, and BigQuery uses AEAD decryption to unwrap the DEK with the KEK only when an authorized analyst needs the protected column. BigQuery column-level security adds a second boundary through policy tags enforced with the Fine-Grained Reader role. The secured-warehouse taxonomy creates a hierarchy where higher access implies lower access:

Columns without tags remain available to all users with warehouse access. Even after re-identification, a column cannot be read until the user is explicitly granted the matching tag.

2.7Regulatory Guardrails for Sensitive Data Protection Jobs

Regulatory guardrails ensure de-identification outputs and surrounding metadata do not recreate exposure under regimes such as HIPAA, GDPR, and CCPA. Sensitive Data Protection job outputs (opens in a new tab) must be written to storage targets inside the secure environment, and Virtual Private Cloud Service Controls (VPC Service Controls) places sensitive data in its own service perimeter to control movement across the perimeter boundary. Under HIPAA practices, the organization signs a Business Associate Agreement (BAA), disables unsupported services, avoids pre-GA offerings for PHI, and never places PHI or credentials in resource metadata, metric labels, log fields, agent definitions, or headers because logs and metadata can be widely visible. Sensitive data therefore flows only from approved inputs to approved perimeters, keys, and masked columns, with audit logging retained for compliance review.

Enforcing data residency and sovereignty requirements means keeping data in an approved physical location and ensuring that storing, encrypting, and accessing that data follows the laws of that location. In Google Cloud, residency is enforced by choosing where data rests, where keys are held, and where users and administrators can reach the data. A compliant design combines regional resource placement, centralized policy guardrails, and localized processing so personally identifiable information (PII) is not transferred across borders without authorization.

3.1Define Regional Granularity and Residency Scope

Regional granularity is the decision about how tightly an application and its data must be pinned to specific zones, regions, or multi-regions. Data residency involves keeping PII within a particular region where it is processed under that region's regulations, which can come from data locality laws or customer contracts such as a mandate to keep government data in-country. Administrators must know the physical location of data centers, services, and resources in the project because the location determines which regulations apply and whether a service-level agreement (SLA) allows that placement. Answer these scoping questions before choosing products and regions:

• Resilience tradeoff: Can the design use multi-regional services to gain resilient, highly available architecture, or must every component stay in one region?
• Access restrictions: Does access to the application have location restrictions that require IP and geo-based filtering or context-based perimeter security?
• Storage restriction: Is data at rest restricted to a specific region, requiring managed services that can pin datasets, buckets, and backups to that region?
• Product availability: What regions can each product actually run in, since some Google Cloud products have no regional restriction and must be validated against product location pages?

3.2Restrict Resource Creation with Organization Policies

The Organization Policy Service is the centralized control that prevents resources from being created outside compliant locations. Policies attach at the organization, folder, or project level of the resource hierarchy, so a restriction set at the top flows down to every child project unless explicitly overridden. This hierarchy dependency means one policy can block an entire estate from deploying storage or processing in a non-compliant geography. The location-restricted resources constraint defines where location-based resources can be created by listing allowed or denied values. Policies can allow or deny multi-regions such as asia and europe, regions such as us-east1 or europe-west1, and individual zones such as europe-west1-b. A companion constraint, domain-restricted sharing, limits which Cloud Identity or Google Workspace customer identities can be added to Identity and Access Management (IAM) policies, so only approved organizational identities gain access to sovereign data.

3.3Localize Storage and Processing in Cloud Storage and BigQuery

Localized storage and processing means Cloud Storage (opens in a new tab) buckets, BigQuery datasets, and the compute that reads them all reside in the same approved geography. When resources are configured in selected regions, Google stores data at rest only in those regions under the Service Specific Terms, including backups when custom backup locations are used. Cloud SQL (opens in a new tab) follows the same model: select the instance region at creation or by editing the instance, and use a cross-region read replica only when the replica region is itself a designated compliant region. Cloud Storage bucket location controls the tradeoff between sovereignty and geographic redundancy:

BigQuery (opens in a new tab) enforces the same locality principle on processing pipelines. A BigQuery dataset pinned to a region or multi-region must be colocated with federated sources and analysis clusters, so data flows from the source to BigQuery without leaving the approved boundary. A federated query against Cloud SQL or Spanner requires the source instance in the same location as the dataset, so a dataset in europe-west1 in Belgium requires the Cloud SQL or Spanner instance in Belgium, while a dataset in the US multi-region requires the source in a single US region. A Bigtable app profile must route to a cluster in the same region as a single-region dataset, such as asia-northeast1 in Tokyo, or to a cluster inside the same multi-region, such as us-west1 in Oregon for a US dataset. Colocate Dataproc Spark clusters using the BigQuery connector and Vertex AI Workbench notebook instances with the datasets they query, and restrict future dataset creation with the Organization Policy Service resource locations constraint for BigQuery. When single-region restriction removes cross-region redundancy, use two single-region buckets with dual-write or with a Cloud Storage copy from the first bucket to the second for stronger recovery point and recovery time assurance. If Google Cloud alone cannot meet distance rules, adopt a multi-cloud or hybrid copy in the same geographic area or use customer-controlled encryption as a workaround only where locality rules permit encrypted out-of-region storage.

3.4Control Encryption Keys and Access Boundaries

Encryption and access control determine who can read sovereign data even if storage location is correct. Cloud External Key Manager (Cloud EKM) lets BigQuery and Compute Engine encrypt data with keys stored and managed in a third-party key management system deployed outside Google infrastructure. Customer-supplied encryption keys (CSEK) let Cloud Storage and Compute Engine protect Google-generated data keys with a key supplied for each operation, which Google purges from its servers after the operation completes and never permanently stores or manages. External key control creates a sovereignty boundary because data cannot be decrypted without keys held outside Google, but it introduces latency, reliability, and recovery dependencies on the external key manager. Keys are integral to writes, so no data can be safely written without them; recovery point objective is less critical than recovery time objective because loss of keys blocks both decryption and safe writes until highly available key service and recovery processes restore access. Network access boundaries complete the design: Google Cloud Armor enforces IP and geo-based constraints on who can reach the application, while VPC Service Controls provides context-based perimeter security and can restrict Cloud SQL import and export through the Cloud SQL Admin API or Cloud Storage API to approved perimeters.

Legal and regulatory compliance ties together three controls: cryptographic control proves who can decrypt data, privacy transformation limits what sensitive values analysts and applications can see, and residency control limits where data and keys can exist and be reached. CMEK with Cloud KMS and audit logging provides revocable ownership, Sensitive Data Protection with tokenization and column controls preserves utility without exposing raw PII or PHI, and organization policies with localized BigQuery and Cloud Storage placement plus external key and perimeter boundaries keep processing inside approved geographies.

Key Facts

• Envelope encryption protects data with a data encryption key (DEK) that is wrapped by a key encryption key (KEK) in Cloud KMS, so disabling or destroying the CMEK makes the protected data inaccessible.
• Configuring CMEK requires creating a Cloud KMS key ring and key, granting the service agent the CryptoKey Encrypter/Decrypter role, and attaching the key at resource creation in the same location.
• Sensitive Data Protection preserves analytic utility through tokenization, masking and redaction, generalization and bucketing, and deterministic or format-preserving cryptographic de-identification.
• BigQuery column-level security uses policy tags where 3_Confidential grants 3_Confidential, 2_Private, and 1_Sensitive, 2_Private grants 2_Private and 1_Sensitive, and 1_Sensitive grants only 1_Sensitive.
• The Organization Policy Service resource locations constraint blocks creation of resources in denied locations but does not move existing data, and BigQuery federated sources must be colocated with the dataset location.