Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Dataplex Lake Zones with BigLake Security Controls
Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An enterprise data engineering team is setting up a centralized data mesh governance architecture across several Google Cloud projects. They have raw and curated data stored in multiple Cloud Storage buckets and BigQuery datasets.
The team must fulfill the following operational and governance requirements:
- Group storage resources into logical domains and automatically harvest schema definitions and partitions without manual pipeline ingestion.
- Register discovered schemas into a centralized catalog and enable analytical queries with column-level masking and row-level access control on Cloud Storage files.
- Enrich discovered data assets with custom, reusable metadata templates containing business classification, data owner contact info, and PII tags to facilitate enterprise searchability.
- Manage and propagate unified access control across the logical hierarchy down to the underlying storage resources.
Which solution should the data engineering team implement?
Explanations
Create Dataproc Metastore clusters for each project, run Cloud Data Loss Prevention (Cloud DLP) discovery inspection jobs to write schemas into BigQuery, configure BigQuery authorized views on Cloud Storage external tables, and apply Data Catalog tag templates using Cloud Functions.
Create Dataproc Metastore clusters for each project, run Cloud Data Loss Prevention (Cloud DLP) discovery inspection jobs to write schemas into BigQuery, configure BigQuery authorized views on Cloud Storage external tables, and apply Data Catalog tag templates using Cloud Functions.
What This Option Proposes
This approach suggests using standalone Dataproc Metastore instances alongside custom Cloud Data Loss Prevention (Cloud DLP) inspection jobs, Cloud Functions, and BigQuery authorized views to discover schemas and apply metadata tags.
Why It Doesn't Meet the Requirement
- Automated Discovery and Registration: Cloud DLP is built for identifying and de-identifying sensitive data patterns rather than acting as an automated structural schema discovery and partition harvesting engine for data lakes.
- Fine-Grained Access on Object Storage: Standard BigQuery authorized views over regular Cloud Storage external tables do not provide native BigLake security capabilities (such as policy tags and dynamic data masking directly on object storage).
- Operational Overhead: Orchestrating metadata tagging through bespoke Cloud Functions and configuring separate metastores across projects creates immense administrative overhead compared to managed Dataplex lake and zone management.
Key Characteristics and Limitations
- Lacks automatic schema harvesting and synchronization across Cloud Storage partitions.
- Fails to propagate unified IAM policies from logical domains down to physical storage assets.
- Multiplies infrastructure costs by running dedicated metastore clusters per project.
This Answer Would Be Correct If
The organization required deep payload PII inspection and custom redaction pipelines for unmanaged external databases rather than an integrated data lake discovery and governance fabric.
Additional Context
While Dataproc Metastore can integrate with Dataplex for Spark execution engines, Dataplex Discovery itself handles schema harvesting natively without requiring dedicated DLP jobs or custom Cloud Functions.
Configure Dataplex lakes and zones, register the Cloud Storage buckets as external table assets without upgrading to BigLake, write custom Python ingestion scripts using the Dataplex Catalog API to generate schema entries, and grant the Dataplex Developer role at the project level.
Configure Dataplex lakes and zones, register the Cloud Storage buckets as external table assets without upgrading to BigLake, write custom Python ingestion scripts using the Dataplex Catalog API to generate schema entries, and grant the Dataplex Developer role at the project level.
What This Option Proposes
This configuration sets up basic Dataplex lakes and zones but leaves the Cloud Storage assets as standard external tables, uses custom Python ingestion scripts for schema cataloging, and applies the roles/dataplex.developer role at the project level.
Why It Doesn't Meet the Requirement
- Lacks Fine-Grained Security: Standard Cloud Storage external tables do not support BigQuery column data masking or row-level access control. The assets must be upgraded to BigLake tables to enforce these security controls.
- Unnecessary Custom Scripting: Dataplex has built-in Discovery that automatically scans assets, extracts schemas, and registers entries in Dataplex Universal Catalog, eliminating the need for custom Python API scripts.
- Incorrect IAM Role Assignment: The
roles/dataplex.developerrole is intended for running analytics workloads/tasks, not managing lake security. Furthermore, data roles must be granted at the lake, zone, or asset level rather than inherited from project-level assignments to propagate correctly.
Key Characteristics and Limitations
- External tables without BigLake support lack fine-grained column masking.
- Custom ingestion scripts introduce code maintenance overhead and lag behind data updates.
- Project-level data roles are not inherited down to Dataplex lake resources.
This Answer Would Be Correct If
You were integrating unsupported non-Google Cloud third-party data sources that required bespoke metadata scrapers via custom connectors.
Additional Context
In Dataplex, data roles set at the project level are not further inherited by lakes in the project; security policies and data roles must be assigned directly at the lake level or lower.
Create Cloud Storage transfer jobs to consolidate all data into a single centralized BigQuery dataset, configure Data Catalog custom entry groups, attach Data Catalog tag templates to the tables, and assign the BigQuery Admin role at the project level.
Create Cloud Storage transfer jobs to consolidate all data into a single centralized BigQuery dataset, configure Data Catalog custom entry groups, attach Data Catalog tag templates to the tables, and assign the BigQuery Admin role at the project level.
What This Option Proposes
This strategy relies on consolidating all data physically into a single BigQuery dataset via Storage Transfer Service / data transfer jobs, managing metadata solely with custom entry groups, and granting project-level BigQuery Admin permissions.
Why It Doesn't Meet the Requirement
- Eliminates Data Lake Architecture: Physically moving and duplicating all Cloud Storage data into native BigQuery tables violates data lake principles and dramatically increases storage and ingestion costs.
- No Automated Lake Discovery: It relies on ETL ingestion rather than Dataplex automated schema and partition discovery across multi-project Cloud Storage assets.
- Violates Principle of Least Privilege: Granting
roles/bigquery.adminat the project level gives overly broad administrative permissions rather than fine-grained, propagated lake/zone/asset access control. - Manual Catalog Overhead: Creating custom entry groups for native Google Cloud resources bypasses automated entry harvesting.
Key Characteristics and Limitations
- Increases operational latency and storage duplication costs.
- Prevents open-format tools (e.g., Spark, Dataproc) from accessing raw Cloud Storage objects governed by unified access policies.
- Lacks fine-grained logical zone separation (raw vs. curated).
This Answer Would Be Correct If
The organization intended to completely deprecate Cloud Storage data lakes and operate exclusively on a centralized, proprietary data warehouse without multi-engine access requirements.
Additional Context
Dataplex allows organizations to keep data distributed across various Cloud Storage buckets and BigQuery datasets while governing them under a unified logical lake hierarchy.
Create a Dataplex lake with curated and raw zones, attach the Cloud Storage buckets and BigQuery datasets as assets, enable Discovery to harvest schemas, upgrade the Cloud Storage assets to BigLake tables, define custom aspect types to enrich catalog entries, and assign Dataplex data and metadata roles at the lake or zone level.
Create a Dataplex lake with curated and raw zones, attach the Cloud Storage buckets and BigQuery datasets as assets, enable Discovery to harvest schemas, upgrade the Cloud Storage assets to BigLake tables, define custom aspect types to enrich catalog entries, and assign Dataplex data and metadata roles at the lake or zone level.
What Google Cloud Dataplex and Universal Catalog Are
Google Cloud Dataplex is an intelligent data fabric that enables organizations to centrally manage, monitor, and govern distributed data across Cloud Storage buckets and BigQuery datasets. Within Dataplex, data is organized logically into lakes, zones (such as raw or curated zones), and assets pointing to physical cloud storage resources. Dataplex Universal Catalog (incorporating Data Catalog capabilities) provides automated discovery, metadata harvesting, and schema management.
How it Meets the Requirements
- Automated Schema Harvesting and Registration: When storage resources are registered as Dataplex assets, Dataplex Discovery automatically crawls files and datasets, infers table schemas and partition definitions, and publishes metadata entries into BigQuery datasets and the catalog.
- Fine-Grained Security on Data Lake Storage: Upgrading Cloud Storage assets to use BigLake tables replaces standard external tables with BigLake-managed tables. This unlocks fine-grained access controls including column-level security, column data masking, and row-level security on data residing in Cloud Storage.
- Metadata Classification and Enrichment: Defining aspect types (the evolutionary metamodel corresponding to Data Catalog tag templates) allows data stewards to attach custom, strongly typed attributes (e.g., PII indicators, data ownership, retention periods) to discovered entries to power faceted search.
- Unified Access Propagation: Dataplex translates and automatically propagates data roles (such as
roles/dataplex.dataReaderandroles/dataplex.dataWriter) and metadata roles (roles/dataplex.metadataReader) down the lake and zone hierarchy to underlying BigQuery and Cloud Storage IAM policies.
Key Advantages
- Zero Data Movement: Manages distributed data in place without copying or consolidating files.
- Automated Metadata Cataloging: Removes the maintenance overhead of manual schema registration.
- Centralized Security Enforcement: Simplifies policy management by managing IAM at logical domain boundaries.
- Enterprise Discoverability: Standardizes metadata tagging across structured and semi-structured assets.
Why This Is Optimal
This architecture leverages fully managed native Google Cloud services designed specifically for automated data lake governance, combining unified access inheritance with automated schema discovery and fine-grained BigLake security.
Try 10 interactive questions like this one, free
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
- 1,353 practice questions across all 5 exam domains
- Every answer explained: why it is right, and why the others are not
- Scout, an AI tutor that answers your follow-up questions
- Readiness tracking by domain, so you know when to book the exam
Related questions
Keep the momentum going with these hand-picked practice scenarios
AdvancedProfessional Data Engineer
Cloud Storage Lifecycle and BigQuery Partitioning
Managing the lake (configuring data discovery, access, and cost controls)
AdvancedProfessional Data Engineer
BigQuery Row Access Policy Grantee Scope
Managing the lake (configuring data discovery, access, and cost controls)
AdvancedProfessional Data Engineer
Security Command Center Event Threat Detection
Monitoring the data lake
AdvancedProfessional Data Engineer
Cloud SQL PostgreSQL WAL Compression
Data replication and failover (e.g., Cloud SQL, Redis clusters)
AdvancedProfessional Data Engineer
BigQuery ML Remote Models with Vertex AI
AI data enrichment
IntermediateProfessional Cloud Architect
Instance Template and Regional Managed Instance Group
Infrastructure orchestration, resource configuration, and patch management
Last updated Validated