professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.
Last updated
An enterprise analytics team orchestrates daily data pipeline workflows in Cloud Composer to refresh BigQuery datasets and publish operational dashboards. The data engineering team needs to operationalize these reporting pipelines under strict governance requirements:
Which solution should the team implement?
Enable the Data Lineage API and activate Dataplex data lineage integration on the Cloud Composer environment. Use standard BigQuery operators for automated lineage tracking, define explicit inlets and outlets with BigQueryTable entities for custom PythonOperator tasks, and assign the roles/composer.worker role to the environment's user-managed service account.
Enable Dataplex data lineage integration, write custom Bash scripts within DAGs to push metadata records directly to the Airflow Cloud SQL metadata database, and grant the environment service account roles/cloudsql.admin access.
Configure a custom Airflow Lineage Backend pointing to Cloud Logging, parse task logs using Cloud Functions to extract SQL queries, and assign the roles/editor basic role to the Cloud Composer service account.
Deploy DAGs using only BashOperator invoking the bq CLI tool, configure Airflow task execution settings with execute_tasks_new_python_interpreter = True, and assign roles/owner to the default Compute Engine service account.
Enable the Data Lineage API and activate Dataplex data lineage integration on the Cloud Composer environment. Use standard BigQuery operators for automated lineage tracking, define explicit inlets and outlets with BigQueryTable entities for custom PythonOperator tasks, and assign the roles/composer.worker role to the environment's user-managed service account.
This solution establishes automated metadata tracking, governance, and secure pipeline orchestration by combining Cloud Composer, Dataplex Universal Catalog, and the Data Lineage API using native OpenLineage integration.
StudyCerts
StudyCerts--enable-cloud-data-lineage-integration) automatically activates the preinstalled apache-airflow-providers-openlineage library. Standard operators like BigQueryInsertJobOperator automatically generate and emit lineage events to Dataplex without requiring custom code.inlets and outlets attributes of PythonOperator tasks with BigQueryTable objects (or AUTO for upstream dependencies), custom execution code explicitly registers lineage relations in the catalog.roles/composer.worker) role to a dedicated user-managed service account provides the minimum IAM permissions required for environment operation, PyPI package handling, and Data Lineage API write operations.This architecture leverages fully managed Google Cloud services to achieve automated pipeline tracking with zero custom backend maintenance. It guarantees strict governance while maintaining robust operational SLAs for analytical reporting.
Enable Dataplex data lineage integration, write custom Bash scripts within DAGs to push metadata records directly to the Airflow Cloud SQL metadata database, and grant the environment service account roles/cloudsql.admin access.
Configure a custom Airflow Lineage Backend pointing to Cloud Logging, parse task logs using Cloud Functions to extract SQL queries, and assign the roles/editor basic role to the Cloud Composer service account.
Deploy DAGs using only BashOperator invoking the bq CLI tool, configure Airflow task execution settings with execute_tasks_new_python_interpreter = True, and assign roles/owner to the default Compute Engine service account.