A data engineering team is orchestrating a multi-step ETL pipeline in Google Cloud Composer using Apache Airflow. The workflow includes conditional execution logic: An ingestion check task uses BranchPythonOperator to determine if new source records are present in Cloud Storage. If data exists, the DAG executes an extraction task followed by a transformation task. If no data exists, the DAG executes a logging task indicating no records were found. A final summary notification task must execute after either branch completes to publish pipeline metrics, provided no executed tasks failed. During testing, when no records exist, the logging task succeeds, but the downstream summary notification task is automatically marked as skipped and never runs. How should the team modify the DAG configuration to ensure the summary notification task executes as required?

Set the `trigger_rule` parameter on the summary notification task to `TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS`: ### What `TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS` is In Apache Airflow orchestrations within **Cloud Composer**, task execution dependencies are governed by trigger rules. By default, tasks use the `all_success` trigger rule, which requires every direct upstream parent task to reach the `success` state before downstream tasks can start. ### How it meets the requirements - **Handles conditional branch skipping**: When a `BranchPythonOperator` selects a path, unselected branch tasks are placed in a `skipped` state. - **Resolves skipped status on convergence**: With `TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS`, the downstream summary notification task will trigger as long as all upstream parent tasks have finished, no upstream task has failed (`failed` or `upstream_failed`), and at least one direct parent task succeeded. - **Maintains error sensitivity**: If the active processing branch fails, the notification task will not erroneously run under success assumptions, respecting pipeline integrity. ### Key advantages and capabilities - **Enables clean branch convergence**: Safely joins divergent branches in a DAG without requiring custom polling or dummy bridging hacks. - **Preserves pipeline status accuracy**: Ensures skipped tasks do not cascade unintentional skips to final operational and monitoring steps. - **Supports declarative flow control**: Keeps conditional DAG logic native, readable, and standard across Cloud Composer environments. ### Why this is optimal This approach directly solves the default Airflow cascade skip behavior associated with branching operators without compromising failure detection or adding redundant operator overhead.

Cloud Composer Dag Branching Trigger Rules | professional-cloud-data-engineer

Cloud Composer Dag Branching Trigger Rules

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

A data engineering team is orchestrating a multi-step ETL pipeline in Google Cloud Composer using Apache Airflow. The workflow includes conditional execution logic:

An ingestion check task uses BranchPythonOperator to determine if new source records are present in Cloud Storage.
If data exists, the DAG executes an extraction task followed by a transformation task.
If no data exists, the DAG executes a logging task indicating no records were found.
A final summary notification task must execute after either branch completes to publish pipeline metrics, provided no executed tasks failed.

During testing, when no records exist, the logging task succeeds, but the downstream summary notification task is automatically marked as skipped and never runs.

How should the team modify the DAG configuration to ensure the summary notification task executes as required?

Set the trigger_rule parameter on the summary notification task to TriggerRule.ALL_FAILED and remove upstream task linkages

Wrap the conditional branches inside an external SubDAG and leave the parent DAG trigger rule as TriggerRule.ALL_SUCCESS

Set the trigger_rule parameter on the summary notification task to TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS

Configure the BranchPythonOperator to return the task IDs of both branches simultaneously and enable depends_on_past=True on the downstream task

Professional Cloud Data Engineer

Cloud Composer Dag Branching Trigger Rules

Related questions