Home Microsoft Exams Google Cloud Exams

Loading page

Curious about new developments and updates? Follow our social media.

Get a free certification question every week.

Microsoft Exams

AZ-104 Administrator
AZ-204 Developer
AZ-900 Fundamentals
All Microsoft Exams

Google Cloud Exams

Associate Data Practitioner
All Google Cloud Exams

Terms Privacy Cookies About

© 2026 StudyCerts. All rights reserved.

Loading page

Ephemeral Dataproc Spark Batch Pipeline | professional-cloud-data-engineer | StudyCerts

Home
Google Cloud Exams
Professional Cloud Data Engineer
Batch
A financial analytics company is migrating its daily batch d…

Professional Cloud Data Engineer

professional-cloud-data-engineer

Advanced

Prepare and test your skills

Track Readiness

Description
Sample Question
Free Trial
Practice Test

Advanced

Prepare and test your skills

Track Readiness

Description
Sample Question
Free Trial
Practice Test

Ephemeral Dataproc Spark Batch Pipeline

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

A financial analytics company is migrating its daily batch data pipeline to Google Cloud. The workload has the following characteristics and constraints:

The current pipeline consists of extensive data transformations written in Apache Spark (PySpark and Scala).
The batch process runs once per day during an off-peak two-hour window to ingest transaction files from Cloud Storage and write structured results to BigQuery.
The engineering team wants to minimize code refactoring effort and avoid the maintenance and cost of idle infrastructure during non-processing hours.

Which execution framework and architecture should the company select?

Execute the existing Spark jobs on ephemeral Cloud Dataproc clusters orchestrated by Cloud Composer, creating clusters on demand and deleting them immediately after job completion

Provision a persistent 24/7 Cloud Dataproc cluster configured with HDFS and autoscaling to handle the nightly Spark batch jobs

Rewrite the data pipelines using the Apache Beam SDK and execute them as batch jobs on Cloud Dataflow

Load raw transaction files directly into BigQuery staging tables and rewrite the transformation logic as BigQuery SQL stored procedures

Explanations

Related questions

Keep the momentum going with these hand-picked practice scenarios

AdvancedProfessional Cloud Data Engineer

A data engineering team maintains a nightly batch Apache Beam pipeline running on Cloud Dataflow that aggregates multi-terabyte transactional logs. The pipeline groups data by merchant ID using a GroupByKey transform and writes aggregated metrics into BigQuery. Monitoring shows the following bottlenecks during job execution: The GroupByKey transform suffers from severe stragglers due to extreme key skew caused by a few high-volume merchants, leaving most worker vCPUs idle while a single worker runs out of memory. Worker nodes encounter severe local Persistent Disk I/O bottlenecks and disk thrashing during intermediate shuffle phases. Compute costs for the nightly batch window are exceeding the allocated budget. Which combination of optimization strategies should the team implement to resolve the performance bottlenecks and minimize execution costs?

AdvancedProfessional Cloud Data Engineer

A data engineering team manages a daily batch pipeline orchestrated by Cloud Composer that processes terabytes of transactional log files from Cloud Storage into partitioned BigQuery tables. Recently, transient worker pod evictions and network timeouts have triggered task retries, resulting in duplicate data rows in BigQuery and failed task executions caused by exhausted worker disk space. You need to re-architect the workflow orchestration to ensure fault tolerance, prevent duplicate records during task retries (idempotency), and prevent worker resource exhaustion. Which architectural strategy should you implement?

AdvancedProfessional Cloud Data Engineer

A data engineering team is deploying automated machine learning pipelines with BigQuery ML and Vertex AI. To satisfy strict governance and reproducibility requirements across iterative model runs, the team must implement an architecture that: Captures immutable, point-in-time versions of training datasets from rapidly changing BigQuery tables Automatically tracks the lineage and linkage across dataset artifacts, hyperparameters, and model evaluations Detects training-serving skew by evaluating production inference data against baseline training statistics Which combination of Google Cloud services and practices should the team implement?

AdvancedProfessional Cloud Data Engineer

An enterprise runs a mission-critical Cloud SQL for PostgreSQL database deployed across multiple Google Cloud regions: Primary instance (db-central): Deployed in us-central1 with High Availability (HA) enabled. Disaster recovery replica (db-west): Cross-region read replica in us-west1. Reporting replica (db-east): Cross-region read replica in us-east1. Application tier: Connects to the database cluster via the Cloud SQL DNS write endpoint. A catastrophic outage renders the entire us-central1 region unavailable. You need to execute an emergency failover to promote db-west to the new primary while minimizing data loss, redirecting application traffic with minimal downtime, and restoring read and HA topology. Which operational procedure should you execute?

AdvancedProfessional Cloud Data Engineer

An enterprise is centralizing sensitive customer transaction data from multiple business units into a shared BigQuery data warehouse. The data engineering and security teams must implement a robust security architecture that satisfies the following requirements: Data Exfiltration Mitigation: Prevent unauthorized movement, copying, or API-based access to the BigQuery datasets from outside the organization or across unapproved projects. Tenant & Regional Isolation: Ensure analysts can only query table rows that match their specific geographic region without creating and managing separate physical tables or authorized views for every team. Field-Level Confidentiality: Automatically mask sensitive columns (such as credit card numbers and credit limits) for standard analysts, while allowing authorized compliance officers with explicit fine-grained permissions to view the cleartext data. Which combination of Google Cloud security mechanisms should the team implement?

IntermediateProfessional Cloud Architect

An organization is designing an automated, event-driven machine learning workflow on Google Cloud. The architecture must satisfy the following technical requirements: Event-Driven Trigger: Automatically initiate the pipeline whenever a new training dataset is uploaded to a designated Cloud Storage bucket. Serverless Orchestration & Training: Run custom distributed model training scripts in custom containers using high-demand NVIDIA A100/H100 GPU accelerators via Vertex AI. Capacity Management & Cost Optimization: Accommodate GPU capacity constraints gracefully by queuing jobs and utilizing Dynamic Workload Scheduler (DWS) with a Flex Start scheduling strategy rather than failing immediately when on-demand quota is constrained. Automated Artifact Management: Upload model artifacts and deploy the trained model to a Vertex AI Endpoint. Which architectural approach should you implement?

Want more questions like this?

Get a free certification question every week.

Are you ready for the Professional Cloud Data Engineer?

50 questions in the full practice test

No account required · Instant results

More Batch practice questions

Get a free certification question with a full explanation every week.

Professional Cloud Data Engineer

Provider: Google Cloud
Passing score: 70%
Questions: 50
Duration: 90 minutes

Other sections in this exam

Designing data processing systems
Ingesting and processing the data
Storing the data
Preparing and using data for analysis
Maintaining and automating data workloads

Ask Scout

Still curious? Scout, our AI tutor, can explain this concept further and answer your follow-up questions.

Batch

Preparing data for feature engineering, training and serving machine learning models (e.g., BigQueryML)

Data replication and failover (e.g., Cloud SQL, Redis clusters)

Identity and Access Management (e.g., Cloud IAM and organization policies)

Using AI Hypercomputer (e.g., using AI Hypercomputer, Cloud Run functions, and Vertex AI for ML/AI workloads; integrating GPUs and TPUs in ML model training and serving; optimizing for different consumption models; and running large-scale AI model training