professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.
Last updated
A data engineering team is orchestrating a multi-step ETL pipeline in Google Cloud Composer using Apache Airflow. The workflow includes conditional execution logic:
BranchPythonOperator to determine if new source records are present in Cloud Storage.During testing, when no records exist, the logging task succeeds, but the downstream summary notification task is automatically marked as skipped and never runs.
How should the team modify the DAG configuration to ensure the summary notification task executes as required?
Configure the BranchPythonOperator to return the task IDs of both branches simultaneously and enable depends_on_past=True on the downstream task
Set the trigger_rule parameter on the summary notification task to TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS
Set the trigger_rule parameter on the summary notification task to TriggerRule.ALL_FAILED and remove upstream task linkages
Wrap the conditional branches inside an external SubDAG and leave the parent DAG trigger rule as TriggerRule.ALL_SUCCESS
Configure the BranchPythonOperator to return the task IDs of both branches simultaneously and enable depends_on_past=True on the downstream task
Set the trigger_rule parameter on the summary notification task to TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS
TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS isIn Apache Airflow orchestrations within Cloud Composer, task execution dependencies are governed by trigger rules. By default, tasks use the all_success trigger rule, which requires every direct upstream parent task to reach the success state before downstream tasks can start.
BranchPythonOperator selects a path, unselected branch tasks are placed in a skipped state.TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS, the downstream summary notification task will trigger as long as all upstream parent tasks have finished, no upstream task has failed (failed or upstream_failed), and at least one direct parent task succeeded.This approach directly solves the default Airflow cascade skip behavior associated with branching operators without compromising failure detection or adding redundant operator overhead.
Set the trigger_rule parameter on the summary notification task to TriggerRule.ALL_FAILED and remove upstream task linkages
Wrap the conditional branches inside an external SubDAG and leave the parent DAG trigger rule as TriggerRule.ALL_SUCCESS