professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
An enterprise is designing a mission-critical streaming data pipeline using Google Cloud Dataflow that ingests high-volume telemetry events and writes them into Google BigQuery.
The architecture must satisfy the following operational requirements:
Which architectural strategy should the data engineering team implement to achieve these requirements?
The BigQuery Storage Write API is an ingestion mechanism in Google Cloud Dataflow that supports high-throughput streaming with exactly-once delivery semantics. When combined with a dead-letter queue (DLQ) pattern and idempotency controls, the pipeline cleanly segregates unprocessable records without interrupting real-time processing streams.
StudyCerts
StudyCerts.getFailedStorageApiInserts() captures failed row insertions along with their specific error messages and payloads. These are transformed and routed to a dedicated dead-letter table in BigQuery or a Pub/Sub topic for alerting and triage.xgooglemessageuid or message_uid) and performing transactional checks or tracking processed IDs in external state, the system ensures that at-least-once message retries do not duplicate records.Alternative approaches that rely on unhandled exceptions cause pipeline worker termination and backlog buildup. Capturing failed mutations via native Dataflow PCollection branching (getFailedStorageApiInserts) provides the highest fidelity, resilience, and operational efficiency.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.