Your financial enterprise processes sensitive customer transaction records subject to strict European Union data residency and sovereignty regulations. The compliance policy mandates the following requirements: All data stored at rest in Cloud Storage must be encrypted using Customer-Managed Encryption Keys (CMEK). Key management cryptographic material and operations must reside exclusively within the europe-west3 (Frankfurt) region. Data processing pipelines running on Cloud Dataflow must execute workers exclusively within europe-west3, while maintaining resilience against potential single-zone capacity constraints or outages. No cross-region network egress or unapproved resource location provisioning is permitted. How should you configure the architecture to enforce compliance and data sovereignty while ensuring job execution resilience?

Create Cloud Storage buckets and a Cloud KMS key ring with keys in the regional location `europe-west3`. Enforce the `gcp.resourceLocations` organization policy for `europe-west3`. Launch the Dataflow job specifying `--region=europe-west3` without setting the `--zone` parameter.: What the solution is This architecture establishes strict regional data residency and governance controls using location-bound Cloud KMS keys, organization policy constraints, and Cloud Dataflow regional job execution. How it meets the requirements Enforcing CMEK per region: Regional storage resources must use a CMEK key ring and key located in the exact same region. Creating the Cloud KMS key ring in europe-west3 satisfies data sovereignty and complies with CMEK location constraints. Preventing cross-region egress and unapproved locations: Enforcing the gcp.resourceLocations organization policy constraint restricted to europe-west3 guarantees that no storage, compute, or cryptographic resources can be provisioned outside the authorized boundary. Ensuring execution resilience without violating residency: Submitting the Dataflow pipeline with --region=europe-west3 and omitting the --zone flag directs Dataflow to route worker VMs dynamically across any healthy zone strictly within europe-west3 based on availability, ensuring resilience against zonal failures while maintaining geographic containment. Key advantages Automated Zonal Failover: Eliminating explicit zonal pinning allows Dataflow's control plane to select optimal zones within europe-west3. Strict Cryptographic Locality: Keys cannot be exported or moved outside the designated europe-west3 regional boundary. Governance Guardrails: The organization policy prevents accidental deployment of resources across regional borders. Why this is optimal This approach directly satisfies EU regulatory data boundaries and CMEK requirements while leveraging regional elasticity for fault tolerance.

Dataflow Residency Cmek Kms Regional Execution | Professional Cloud Data Engineer

AdvancedProfessional Cloud Data Engineer

Your enterprise runs critical batch data processing pipelines orchestrated by Cloud Composer in the us-central1 region. You are tasked with designing a cross-region disaster recovery (DR) architecture that can restore workflow orchestration in us-east4 if an entire region experiences an outage. The design must satisfy the following criteria: Maintain DAG execution history and pipeline state across regions. Ensure configuration consistency and minimize workflow restoration time. Prevent inconsistent data states caused by re-executing tasks that ran after the last backup. Which strategy should you implement?

Running jobs in multiple regions or zones

AdvancedProfessional Cloud Data Engineer

An enterprise processes terabytes of analytics data daily using batch Apache Beam pipelines on Dataflow. The primary input dataset currently resides in a Cloud Storage multi-region bucket located in the us multi-region, while the pipeline's temporary and staging files (gcpTempLocation) are written to a bucket in us-east1. The Dataflow worker compute instances are provisioned to execute in us-central1. The cloud operations team reports substantial inter-region network egress charges and increased execution wall time during pipeline shuffle and staging phases. Additionally, the batch workloads are non-time-sensitive. Which architectural strategy should the data engineering team implement to minimize network egress costs, improve I/O performance, and optimize economic efficiency?

Running jobs in multiple regions or zones

AdvancedProfessional Cloud Data Engineer

An enterprise data platform spans multiple Google Cloud projects where distributed teams produce BigQuery analytical datasets and orchestrate machine learning pipelines in Vertex AI. The central governance team needs to implement a unified federated discovery and metadata management solution. The solution must meet the following requirements: Enable centralized search and discovery of data assets and machine learning pipeline artifacts across all projects Automatically visualize upstream root causes and downstream impacts through end-to-end lineage graphs Prevent naming collisions across distributed projects without centralizing the physical storage Which solution should you implement?

Building a federated governance model for distributed data systems

AdvancedProfessional Cloud Data Engineer

A data engineer is troubleshooting a slow-running SQL query in BigQuery that feeds an executive Looker Studio dashboard. When analyzing the Query Execution Graph and stage performance metrics in the Google Cloud console, the engineer observes the following behaviors in Stage S04 (a hash JOIN operation): The Max slot time (4 minutes 12 seconds) is substantially higher than the Average slot time (14 seconds). A single worker processed 85% of the total read records in the stage, while all other worker slots finished almost immediately. The stage produced an unusually high volume of Bytes shuffled to remote distributed memory. What is the primary root cause of this query performance bottleneck, and what is the optimal optimization strategy to resolve it?

Troubleshooting poor performing queries

AdvancedProfessional Cloud Data Engineer

A data engineer is designing an automated post-ingestion cleansing pipeline in BigQuery for a multi-terabyte event ingestion table named telemetry.raw_events. Due to upstream at-least-once streaming semantics, duplicate event payloads are frequently ingested. The cleansing pipeline must meet the following operational requirements: Deduplicate records sharing the same device_id and event_id. Retain only the most recent record based on ingestion_timestamp. Overwrite the existing table with the cleansed dataset without leaving temporary analytical ranking columns in the final schema. Maximize query processing performance and slot efficiency across large partitions. Which SQL pattern should the data engineer implement?

Data cleansing

FundamentalAssociate Data Practitioner

Cloud Data Fusion

Choose products required to implement basic transformation pipelines

Professional Cloud Data Engineer

Dataflow Residency Cmek Kms Regional Execution

Related questions

Cloud Data Fusion

Professional Cloud Data Engineer

Dataflow Residency Cmek Kms Regional Execution

What the solution is

How it meets the requirements

Key advantages

Why this is optimal

Related questions

Cloud Data Fusion

What the solution is

How it meets the requirements

Key advantages

Why this is optimal