Home Microsoft Exams Google Cloud Exams

Loading page

Curious about new developments and updates? Follow our social media.

Get a free certification question every week.

Microsoft Exams

AZ-104 Administrator
AZ-204 Developer
AZ-900 Fundamentals
All Microsoft Exams

Google Cloud Exams

Associate Data Practitioner
All Google Cloud Exams

Terms Privacy Cookies About Pricing

© 2026 StudyCerts. All rights reserved.

StudyCerts is an independent service and is not affiliated with, endorsed by, or sponsored by any certification body or cloud provider.

Loading page

Home
Google Cloud Exams
Professional Cloud Data Engineer
Preparing for data corruption and missing data
An enterprise ingestion pipeline processes streaming transac…

Professional Cloud Data Engineer

professional-cloud-data-engineer

Advanced

Prepare and test your skills

Description
Sample Question
Free Trial
Practice Test

Advanced

Prepare and test your skills

Description
Sample Question
Free Trial
Practice Test

Automated Pipeline Recovery Pubsub Gcs Composer

Section: Preparing for data corruption and missing data

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

...

...

...

...

...

Explanations

...

View source documentation

Related questions

Keep the momentum going with these hand-picked practice scenarios

AdvancedProfessional Cloud Data Engineer

An e-commerce company ingests high-throughput user clickstream events via Cloud Pub/Sub and processes them using a streaming Dataflow pipeline before persisting the records into BigQuery. Upstream mobile application updates have occasionally introduced unannounced schema changes, unexpected null values in mandatory telemetry attributes, and malformed JSON payloads, causing unhandled pipeline exceptions and data corruption in downstream reporting tables. The data engineering team needs to implement an automated validation framework that detects schema drift and corrupted records in real time, isolates non-conforming payloads without interrupting the processing of valid events, and alerts engineers when anomaly rates exceed predefined thresholds. Which architecture should the team deploy?

Preparing for data corruption and missing data

AdvancedProfessional Cloud Data Engineer

A data engineering team operates an enterprise data pipeline on Google Cloud that ingests transactional records into analytical storage. During recent pipeline runs, upstream schema mismatches and partial extraction failures caused silent data corruption and missing records, which went undetected until downstream reports failed. The team needs to establish a proactive observability and alerting strategy to rapidly detect data completeness, freshness, and consistency anomalies. The strategy must: Track validation statistics, processing latencies, and error patterns emitted during pipeline execution. Automatically trigger notifications to on-call teams when consistency or completeness thresholds are breached. Consolidate time-series pipeline metrics, relevant log entries, and active incident states into a unified view for rapid triage. Which strategy should the team implement?

Preparing for data corruption and missing data

AdvancedProfessional Cloud Data Engineer

An enterprise financial services company is modernizing its mission-critical core banking system on Google Cloud. The system has the following architectural requirements: Support a structured relational schema with strict ACID transaction guarantees and serializable isolation. Provide horizontal write scalability across multiple geographical regions without requiring application-level sharding. Maintain strong external consistency with synchronous multi-region replication. Meet a 99.999% (five-nines) availability Service Level Agreement (SLA). Which database service should the organization select to meet these requirements?

Choosing managed services (e.g., BigQuery, BigLake, AlloyDB, Bigtable, Spanner, Cloud SQL, Cloud Storage, Firestore, Memorystore)

AdvancedProfessional Cloud Data Engineer

A data engineering team operates a 24/7 streaming Cloud Dataflow pipeline written in Java that ingests high-volume JSON event streams from Pub/Sub and writes structured rows into BigQuery. Recently, unexpected schema changes upstream have caused malformed JSON payloads, leading to uncaught runtime parsing exceptions inside a DoFn that cause streaming worker tasks to retry indefinitely and stall the pipeline. The team needs to update the Apache Beam pipeline architecture to validate incoming records, immediately divert uncorrectable corrupt payloads to a dead-letter storage sink for auditing and reprocessing, and ensure that valid data continues to flow to BigQuery without pipeline disruption. Which design pattern should the team implement?

AdvancedProfessional Cloud Data Engineer

An enterprise data architect is designing an automated de-identification pipeline using Sensitive Data Protection before ingesting a sensitive dataset into BigQuery. The source dataset contains four key fields with distinct compliance, system, and analytical constraints: account_number: A fixed 10-digit numeric string that must be ingested by a legacy downstream application requiring strict 10-digit numeric format preservation, while also supporting two-way re-identification by security administrators. user_id: A customer identifier required for multi-table analytical joins across historical datasets; it requires two-way re-identification and maximum cryptographic security without character set or length preservation constraints. user_age: A quasi-identifier that must be generalized into discrete intervals (such as 20–29, 30–39) to reduce re-identification risk under k-anonymity requirements while retaining demographic utility. user_pin: A 4-digit secret that must never be reversible under any circumstances, but must maintain referential integrity across analytical queries. Which combination of Sensitive Data Protection transformational methods should the architect select for each respective column?

Privacy (e.g., strategies to handle personally identifiable information)

IntermediateProfessional Cloud Architect

An e-commerce company runs a microservices-based application across multiple services in Google Cloud. Following a recent feature release, operations engineers notice three distinct issues affecting production: Customers experience intermittent latency spikes during checkout transactions traversing multiple downstream services. Certain backend worker instances exhibit abnormally high CPU and memory consumption. Unhandled application crashes and runtime exceptions are being written to logs without being aggregated. Which combination of Google Cloud Application Performance Management (APM) and observability tools should you implement to address each issue?

Want more questions like this?

Get a free certification question every week.

Last updated August 27, 2026

Are you ready for the Professional Cloud Data Engineer?

50 questions in the full practice test

More Preparing for data corruption and missing data practice questions

Get a free certification question with a full explanation every week.

Professional Cloud Data Engineer

Provider: Google Cloud
Passing score: 70%
Questions: 50
Duration: 90 minutes

Other sections in this exam

Designing data processing systems
Ingesting and processing the data
Storing the data
Preparing and using data for analysis
Maintaining and automating data workloads

Ask Scout

Still curious? Scout, our AI tutor, can explain this concept further and answer your follow-up questions.

An enterprise ingestion pipeline processes streaming transactional data that is staged in Cloud Storage before undergoing complex transformations orchestrated by Cloud Composer. Recent schema deviations and malformed payloads have corrupted downstream analytics tables, leading to pipeline halts and unrecoverable data.

You need to architect an automated recovery and reprocessing workflow that satisfies the following operational requirements:

Automatically quarantine unparseable or corrupted streaming messages without crashing primary pipelines
Enable point-in-time recovery to a clean state if corrupted files are written to Cloud Storage
Automatically orchestrate reprocessing from validated checkpoints while maintaining security governance and execution auditability

Which architecture should you implement?

Configure an exponential backoff retry policy on the primary Pub/Sub subscription, use Cloud Storage Object Retention Locks to prevent overwrite errors, and allow DAGs to query the Cloud Composer Airflow metadata database directly to identify failed tasks.

Configure Pub/Sub message filtering to drop malformed messages immediately, enable Cloud Storage soft delete with daily snapshots, and grant data engineers direct IAM permissions to edit DAG code directly in the Cloud Composer storage bucket during recovery.

Configure a Pub/Sub dead-letter topic to isolate invalid streaming messages, enable Object Versioning on the Cloud Storage staging bucket to restore prior uncorrupted states, and deploy Cloud Composer DAGs via CI/CD to orchestrate automated reprocessing from validated checkpoints.

Deploy an AlloyDB Omni instance to replicate streaming transactions, enable PostgreSQL Write-Ahead Logging (WAL) archiving to Cloud Storage, and execute ad-hoc pg_restore scripts from Cloud Shell to reprocess data.

Configure an exponential backoff retry policy on the primary Pub/Sub subscription, use Cloud Storage Object Retention Locks to prevent overwrite errors, and allow DAGs to query the Cloud Composer Airflow metadata database directly to identify failed tasks.

Configure Pub/Sub message filtering to drop malformed messages immediately, enable Cloud Storage soft delete with daily snapshots, and grant data engineers direct IAM permissions to edit DAG code directly in the Cloud Composer storage bucket during recovery.

Configure a Pub/Sub dead-letter topic to isolate invalid streaming messages, enable Object Versioning on the Cloud Storage staging bucket to restore prior uncorrupted states, and deploy Cloud Composer DAGs via CI/CD to orchestrate automated reprocessing from validated checkpoints.

What the solution is

This architecture combines Google Cloud Pub/Sub dead-letter topics (DLTs), Cloud Storage Object Versioning, and managed Apache Airflow orchestration via Cloud Composer deployed through secure continuous integration and delivery (CI/CD) pipelines.

How it meets the requirements

Quarantine corrupted streaming data: Pub/Sub dead-letter topics automatically route unparseable messages or payloads that exceed maximum delivery attempts away from the main ingestion subscriber into a dedicated quarantine topic, preventing pipeline halts.
Enable point-in-time recovery: Cloud Storage Object Versioning preserves noncurrent versions of overwritten or deleted objects, allowing rapid rollback to an uncorrupted historical state prior to the corrupted write.
Automate reprocessing and maintain auditability: Cloud Composer DAGs orchestrate checkpointed reprocessing and backfilling. Managing and deploying DAGs via CI/CD repositories ensures version-controlled recovery workflows and full operational auditability.

Key advantages and capabilities

Resilience: Prevents bad messages from continuously blocking streaming pipelines (poison pill mitigation).
Non-destructive rollback: Protects staging data lakes from accidental overwrites or downstream batch transformation bugs.
Secure orchestration: CI/CD deployment ensures DAG changes are reviewed and tested before triggering automated remediation workflows.

Deploy an AlloyDB Omni instance to replicate streaming transactions, enable PostgreSQL Write-Ahead Logging (WAL) archiving to Cloud Storage, and execute ad-hoc pg_restore scripts from Cloud Shell to reprocess data.