professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.
Last updated
An enterprise processes terabytes of analytics data daily using batch Apache Beam pipelines on Dataflow. The primary input dataset currently resides in a Cloud Storage multi-region bucket located in the us multi-region, while the pipeline's temporary and staging files (gcpTempLocation) are written to a bucket in us-east1. The Dataflow worker compute instances are provisioned to execute in us-central1.
The cloud operations team reports substantial inter-region network egress charges and increased execution wall time during pipeline shuffle and staging phases. Additionally, the batch workloads are non-time-sensitive.
Which architectural strategy should the data engineering team implement to minimize network egress costs, improve I/O performance, and optimize economic efficiency?
Change the default storage class of the multi-region bucket to Archive storage and keep worker VMs running continuously in us-central1 across all availability zones.
Retain the multi-region storage and cross-region worker configuration, but deploy a cross-region internal Application Load Balancer to dynamically route intermediate shuffle traffic across regions.
Maintain the worker execution in us-central1 and staging in us-east1, but increase the worker VM thread count and maximum workers to mask network transfer latency.
Migrate the input data and configure the temporary staging bucket in a regional Cloud Storage bucket located in us-central1, execute Dataflow workers in us-central1, and enable Flexible Resource Scheduling (FlexRS).
Change the default storage class of the multi-region bucket to Archive storage and keep worker VMs running continuously in us-central1 across all availability zones.
Retain the multi-region storage and cross-region worker configuration, but deploy a cross-region internal Application Load Balancer to dynamically route intermediate shuffle traffic across regions.
Maintain the worker execution in us-central1 and staging in us-east1, but increase the worker VM thread count and maximum workers to mask network transfer latency.
Migrate the input data and configure the temporary staging bucket in a regional Cloud Storage bucket located in us-central1, execute Dataflow workers in us-central1, and enable Flexible Resource Scheduling (FlexRS).
Data locality refers to the architectural design of placing compute processing resources in the exact same geographic region and zone as the storage services they interact with. Flexible Resource Scheduling (FlexRS) is a Dataflow execution option designed for non-time-sensitive batch workloads that reduces compute costs by utilizing a mix of Spot/preemptible virtual machines (VMs) and standard VMs with delayed scheduling.
StudyCerts
StudyCertsgcpTempLocation), and the Dataflow worker VMs within us-central1, all data transfers remain strictly intra-regional. This completely eliminates expensive inter-region cross-network egress charges incurred during stage shuffling and temporary state writes.Co-locating all storage locations (gcpTempLocation and raw data) with the compute region while activating FlexRS provides the highest reduction in cloud billing and network overhead without violating batch execution requirements.