professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.
Last updated
An enterprise data engineering team uses Cloud Composer to orchestrate batch pipelines that compute daily sales aggregates and precalculate key business metrics. The precalculated datasets are stored in Cloud Storage and must be loaded into an external relational reporting database accessed by visualization dashboards.
To ensure data integrity, the pipeline must support automated task retries and backfilling historical dates without creating duplicate aggregate records or causing inconsistent metric states in the target database.
How should the team configure the Dataproc Serverless for Apache Spark Cloud Storage to JDBC template to achieve idempotent updates?
Set the output write mode parameter (MODE) to Append and rely on standard JDBC transaction commits
Keep the default write mode (ErrorIfExists) and manage pipeline retries with exponential backoff
Set the output write mode parameter (MODE) to Ignore and disable dynamic allocation
Set the output write mode parameter (MODE) to Overwrite when submitting the template execution
Set the output write mode parameter (MODE) to Append and rely on standard JDBC transaction commits
Keep the default write mode (ErrorIfExists) and manage pipeline retries with exponential backoff
Set the output write mode parameter (MODE) to Ignore and disable dynamic allocation
Set the output write mode parameter (MODE) to Overwrite when submitting the template execution
The Overwrite write mode in the Dataproc Serverless for Apache Spark Cloud Storage to JDBC template directs the Spark execution engine to replace the existing target table content or target partition data with the newly computed dataset.
StudyCerts
StudyCertsOverwrite ensures that executing the pipeline multiple times for the same time window produces the exact same end state without duplicate records.DELETE or TRUNCATE SQL scripts in Cloud Composer DAGs.Using the built-in Overwrite parameter in the Dataproc Serverless template provides declarative, atomic-like replacement semantics at the Spark execution layer, ensuring reliable idempotent operations during automated retries and backfills.