professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.
Last updated
Your enterprise data platform processes large daily batch and streaming workloads across Dataproc, Dataflow, and BigQuery. In preparation for a planned fivefold surge in data volume during a seasonal retail event, you need to proactively evaluate and manage resource limits to prevent operational bottlenecks and pipeline failures during cluster and worker scale-ups.
Which monitoring and capacity planning strategy should you implement to identify resource and quota constraints before they cause pipeline failures?
Schedule daily Cloud SQL compute scale-up and scale-down operations every two hours during pipeline execution windows to manage memory consumption.
Create Cloud Monitoring dashboards and threshold alert policies tracking YARN Pending and Allocated resources for Dataproc alongside quota allocation metrics for Compute Engine CPUs, while tuning the autoscaling scaleUpFactor.
Disable dynamic allocation across all Spark and MapReduce workloads while setting Cloud SQL active query threshold alerts on long-running transactions.
Configure alerting on Cassandra read and write request 99th-percentile latencies and pod volume utilization thresholds in Cloud Monitoring.
Schedule daily Cloud SQL compute scale-up and scale-down operations every two hours during pipeline execution windows to manage memory consumption.
Create Cloud Monitoring dashboards and threshold alert policies tracking YARN Pending and Allocated resources for Dataproc alongside quota allocation metrics for Compute Engine CPUs, while tuning the autoscaling scaleUpFactor.
This strategy establishes comprehensive, proactive observability across cluster compute demand and Google Cloud resource quotas by monitoring Hadoop YARN metrics and Cloud Quota allocation metrics in Cloud Monitoring.
StudyCerts
StudyCertsPending resource and Allocated resource metrics exposes when pending container requests cannot be fulfilled by available cluster capacity. Setting an appropriate scaleUpFactor (e.g., higher for Spark dynamic allocation, moderate for MapReduce) ensures that autoscaling adds worker nodes responsively.scaleUpFactor with workload characteristics prevents autoscaler thrashing and ensures smooth scaling during traffic spikes.Monitoring quota utilization alongside orchestrator-level resource demand (such as YARN pending containers) directly addresses the root causes of scale-up failures in distributed data pipelines.
Disable dynamic allocation across all Spark and MapReduce workloads while setting Cloud SQL active query threshold alerts on long-running transactions.
Configure alerting on Cassandra read and write request 99th-percentile latencies and pod volume utilization thresholds in Cloud Monitoring.