An enterprise data engineering team is setting up a centralized data mesh governance architecture across several Google Cloud projects. They have raw and curated data stored in multiple Cloud Storage buckets and BigQuery datasets. The team must fulfill the following operational and governance requirements: Group storage resources into logical domains and automatically harvest schema definitions and partitions without manual pipeline ingestion. Register discovered schemas into a centralized catalog and enable analytical queries with column-level masking and row-level access control on Cloud Storage files. Enrich discovered data assets with custom, reusable metadata templates containing business classification, data owner contact info, and PII tags to facilitate enterprise searchability. Manage and propagate unified access control across the logical hierarchy down to the underlying storage resources. Which solution should the data engineering team implement?

Create a Dataplex lake with curated and raw zones, attach the Cloud Storage buckets and BigQuery datasets as assets, enable Discovery to harvest schemas, upgrade the Cloud Storage assets to BigLake tables, define custom aspect types to enrich catalog entries, and assign Dataplex data and metadata roles at the lake or zone level.: ### What Google Cloud Dataplex and Universal Catalog Are **Google Cloud Dataplex** is an intelligent data fabric that enables organizations to centrally manage, monitor, and govern distributed data across **Cloud Storage** buckets and **BigQuery** datasets. Within Dataplex, data is organized logically into **lakes**, **zones** (such as raw or curated zones), and **assets** pointing to physical cloud storage resources. **Dataplex Universal Catalog** (incorporating Data Catalog capabilities) provides automated discovery, metadata harvesting, and schema management. ### How it Meets the Requirements - **Automated Schema Harvesting and Registration**: When storage resources are registered as Dataplex assets, **Dataplex Discovery** automatically crawls files and datasets, infers table schemas and partition definitions, and publishes metadata entries into BigQuery datasets and the catalog. - **Fine-Grained Security on Data Lake Storage**: Upgrading Cloud Storage assets to use **BigLake tables** replaces standard external tables with BigLake-managed tables. This unlocks fine-grained access controls including **column-level security**, **column data masking**, and **row-level security** on data residing in Cloud Storage. - **Metadata Classification and Enrichment**: Defining **aspect types** (the evolutionary metamodel corresponding to Data Catalog **tag templates**) allows data stewards to attach custom, strongly typed attributes (e.g., PII indicators, data ownership, retention periods) to discovered entries to power faceted search. - **Unified Access Propagation**: Dataplex translates and automatically propagates **data roles** (such as `roles/dataplex.dataReader` and `roles/dataplex.dataWriter`) and **metadata roles** (`roles/dataplex.metadataReader`) down the lake and zone hierarchy to underlying BigQuery and Cloud Storage IAM policies. ### Key Advantages - **Zero Data Movement**: Manages distributed data in place without copying or consolidating files. - **Automated Metadata Cataloging**: Removes the maintenance overhead of manual schema registration. - **Centralized Security Enforcement**: Simplifies policy management by managing IAM at logical domain boundaries. - **Enterprise Discoverability**: Standardizes metadata tagging across structured and semi-structured assets. ### Why This Is Optimal This architecture leverages fully managed native Google Cloud services designed specifically for automated data lake governance, combining unified access inheritance with automated schema discovery and fine-grained BigLake security.

Configure Dataplex Discovery Metadata Governance | professional-cloud-data-engineer

Configure Dataplex Discovery Metadata Governance

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An enterprise data engineering team is setting up a centralized data mesh governance architecture across several Google Cloud projects. They have raw and curated data stored in multiple Cloud Storage buckets and BigQuery datasets.

The team must fulfill the following operational and governance requirements:

Group storage resources into logical domains and automatically harvest schema definitions and partitions without manual pipeline ingestion.
Register discovered schemas into a centralized catalog and enable analytical queries with column-level masking and row-level access control on Cloud Storage files.
Enrich discovered data assets with custom, reusable metadata templates containing business classification, data owner contact info, and PII tags to facilitate enterprise searchability.
Manage and propagate unified access control across the logical hierarchy down to the underlying storage resources.

Which solution should the data engineering team implement?

Configure Dataplex lakes and zones, register the Cloud Storage buckets as external table assets without upgrading to BigLake, write custom Python ingestion scripts using the Dataplex Catalog API to generate schema entries, and grant the Dataplex Developer role at the project level.

Create Cloud Storage transfer jobs to consolidate all data into a single centralized BigQuery dataset, configure Data Catalog custom entry groups, attach Data Catalog tag templates to the tables, and assign the BigQuery Admin role at the project level.

Create Dataproc Metastore clusters for each project, run Cloud Data Loss Prevention (Cloud DLP) discovery inspection jobs to write schemas into BigQuery, configure BigQuery authorized views on Cloud Storage external tables, and apply Data Catalog tag templates using Cloud Functions.

Create a Dataplex lake with curated and raw zones, attach the Cloud Storage buckets and BigQuery datasets as assets, enable Discovery to harvest schemas, upgrade the Cloud Storage assets to BigLake tables, define custom aspect types to enrich catalog entries, and assign Dataplex data and metadata roles at the lake or zone level.

Professional Cloud Data Engineer

Configure Dataplex Discovery Metadata Governance

Related questions