A data platform team is managing an enterprise data lake on Google Cloud where petabyte-scale datasets in BigQuery and Cloud Storage receive continuous updates. The team needs to implement an automated data quality and monitoring framework to meet the following requirements: Validate completeness and correctness across incoming daily data records without running costly full-table scans across historical partitions. Detect and profile any unmasked personally identifiable information (PII) introduced by source systems and publish findings directly to the metadata governance catalog. Continuously measure streaming pipeline processing delays against a service level objective (SLO) data freshness target and alert when latency spikes occur. Which architecture should the data platform team deploy to satisfy these requirements?

Configure Dataplex auto data quality scans with an incremental scope tracking the ingestion timestamp column, enable Sensitive Data Protection discovery to export data profiles to Dataplex Universal Catalog, and configure Cloud Monitoring alert policies on Dataflow data freshness metrics.: What the Solution Is This architecture combines Dataplex auto data quality scans, Sensitive Data Protection (Cloud DLP) discovery, and Cloud Monitoring with Dataflow metrics to establish an automated, scalable data lake monitoring framework. How It Meets the Requirements Incremental completeness and correctness validation: Configuring Dataplex data quality scans with an Incremental scope uses a DATE or TIMESTAMP column to evaluate only newly appended data records. This avoids scanning entire multi-terabyte or petabyte tables, drastically reducing compute query costs and processing time. PII detection and catalog integration: Enabling Sensitive Data Protection discovery continuously scans new and modified assets across BigQuery and Cloud Storage to detect sensitive infoTypes, classify risk levels, and automatically export rich data profiles into Dataplex Universal Catalog. Freshness monitoring and alerting: Dataflow natively reports the data_freshness metric (measuring the age of the oldest unprocessed element) directly into Cloud Monitoring, enabling threshold-based alerting against defined freshness SLOs. Key Advantages Cost-optimized evaluation: Incremental scans restrict validation queries to recent delta partitions. Unified metadata governance: Publishes quality scan scores and sensitive data profiles natively to BigQuery and Dataplex Catalog. Automated compliance: Continuously discovers undocumented PII across decentralized lake storage. Real-time observability: Directly tracks pipeline latency Service Level Indicators (SLIs) in Cloud Monitoring. Why This Is Optimal This solution relies entirely on native, fully managed Google Cloud services designed for data governance, automated scanning, and streaming observability, avoiding custom script maintenance while ensuring minimal compute overhead.

Automated Data Quality Freshness Lake Monitoring | Professional Cloud Data Engineer

AdvancedProfessional Cloud Data Engineer

An enterprise operates a multitenant data lake on Google Cloud using Cloud Storage and BigQuery to process sensitive analytics workloads. The security team requires a centralized monitoring strategy to detect unauthorized activities, identify anomalous Identity and Access Management (IAM) privilege escalations, and track defense evasion attempts—such as unauthorized modifications to VPC Service Controls perimeters or disabling secure transport policies on storage buckets. Which solution should you implement to meet these security monitoring and data governance requirements?

Monitoring the data lake

AdvancedProfessional Cloud Data Engineer

An enterprise data lake team runs analytical queries against large data lake datasets in BigQuery. The team notices intermittent performance degradation and wants to establish proactive observability to identify bottlenecks and optimize compute resource allocation. They have the following operational requirements: Create an automated alerting policy in Cloud Monitoring that triggers when severe query latency outliers occur (specifically, when the 99th percentile of query execution times across a 5-minute rolling window exceeds 60 seconds). Track slot consumption alongside query execution metrics on a custom dashboard to evaluate whether to adjust capacity commitments and slot assignments. Which configuration should the team implement in Cloud Monitoring?

Monitoring the data lake

AdvancedProfessional Cloud Data Engineer

Your financial enterprise processes sensitive customer transaction records subject to strict European Union data residency and sovereignty regulations. The compliance policy mandates the following requirements: All data stored at rest in Cloud Storage must be encrypted using Customer-Managed Encryption Keys (CMEK). Key management cryptographic material and operations must reside exclusively within the europe-west3 (Frankfurt) region. Data processing pipelines running on Cloud Dataflow must execute workers exclusively within europe-west3, while maintaining resilience against potential single-zone capacity constraints or outages. No cross-region network egress or unapproved resource location provisioning is permitted. How should you configure the architecture to enforce compliance and data sovereignty while ensuring job execution resilience?

Running jobs in multiple regions or zones

AdvancedProfessional Cloud Data Engineer

A data engineer is designing security controls for an enterprise Looker deployment that queries a central data warehouse. The implementation must satisfy the following governance requirements: Model-Level Access: Users must only be allowed to see and query specific LookML models assigned to their department. Row-Level Security: Within a shared sales Explore, users must only view records associated with their assigned sales region. Field-Level Security: Sensitive financial metrics (such as gross profit margin) within the shared Explore must only be visible to authorized finance personnel. Which combination of Looker access controls and LookML configurations should the data engineer implement?

Publishing reports and visualizations

AdvancedProfessional Cloud Data Engineer

An organization is designing an automated Dataflow ingestion pipeline to de-identify sensitive structured records containing Personally Identifiable Information (PII) before storing them in BigQuery for analytics. The privacy compliance team establishes the following technical constraints: Identifier Tokenization: The 10-digit numerical patient_account_id column must retain referential integrity for analytical joins, remain reversible for authorized re-identification pipelines, and preserve its original 10-digit numeric character set to maintain backward compatibility with legacy schema validators. Quasi-Identifier Generalization: The patient_age column must be abstracted into discrete 10-year cohorts to mitigate re-identification risk while preserving population distribution metrics. Key Security: Plaintext cryptographic keys must not be exposed in configuration templates; all token encryption keys must be encrypted using Cloud Key Management Service (Cloud KMS). Which transformation configuration in Cloud Sensitive Data Protection (Cloud DLP) meets all of these requirements?

Legal and regulatory compliance

FundamentalAssociate Data Practitioner

BigQuery Data Transfer Service

Choose the appropriate extraction tool (e.g., Dataflow, BigQuery Data Transfer Service, Database Migration Service, Cloud Data Fusion)

Professional Cloud Data Engineer

Automated Data Quality Freshness Lake Monitoring

Related questions

BigQuery Data Transfer Service

Professional Cloud Data Engineer

Automated Data Quality Freshness Lake Monitoring

What the Solution Is

How It Meets the Requirements

Key Advantages

Why This Is Optimal

Related questions

BigQuery Data Transfer Service

What the Solution Is

How It Meets the Requirements

Key Advantages

Why This Is Optimal