A financial analytics firm is building an event-driven data ingestion pipeline. Whenever transaction batch files are uploaded to a Cloud Storage bucket, the system must immediately trigger an orchestration process that: Reads configuration metadata from Cloud Firestore. Executes a containerized data-cleaning microservice on Cloud Run. Catches specific HTTP errors (such as 404 Not Found) and retries transient service faults with backoff, while raising unhandled exceptions. The organization requires a fully serverless, low-latency solution that minimizes operational management and isolates orchestration logic from business logic. Which architecture should the data engineering team implement to meet these requirements?

Create an Eventarc trigger filtering for `google.cloud.storage.object.v1.finalized` events that routes directly to a Google Cloud Workflows definition, using built-in Google APIs connectors with `try/except` and `switch` blocks in YAML.: ### What the Solution Is **Google Cloud Workflows** combined with **Eventarc** provides a fully managed, serverless orchestration framework specifically designed for low-latency, event-driven microservice and API coordination. The orchestration state machine is declared in **YAML** or **JSON**, allowing developers to sequence services, manage states, and implement custom error-handling logic. ### How It Meets the Requirements - **Event-Driven & Low Latency**: **Eventarc** captures the `google.cloud.storage.object.v1.finalized` event directly from Cloud Storage and routes it to the target workflow within milliseconds, executing the pipeline on demand without polling delays. - **Integrated Connectors**: Workflows includes built-in connectors (such as `googleapis.firestore.v1` and `googleapis.run.v1`), which handle authentication, request formatting, and API interaction natively. - **Built-in Error Handling & Retries**: Workflows supports structured `try`/`except` constructs, conditional `switch` branching, and automated retry policies with exponential backoff directly within the YAML specification. - **Serverless & Operational Simplicity**: The entire stack scales down to zero when idle, incurring no persistent infrastructure overhead or cluster management. ### Key Advantages - **Decoupled Architecture**: Separates the sequencing, dependency management, and routing logic from individual microservices. - **Robust Exception Handling**: Captures error payloads (e.g., `e.code == 404`) and dynamically directs the execution path or halts with custom messages. - **Zero Server Management**: Eliminates VM patching, cluster scaling, and idle compute costs. ### Why This Is Optimal Unlike container-based orchestrators that introduce node provisioning latency or batch-oriented workflow tools, Google Cloud Workflows starts immediately upon receiving the Eventarc trigger, providing sub-second invocation times and native connector support for Google Cloud APIs.

Serverless Eventarc Workflows Orchestration | professional-cloud-data-engineer

Serverless Eventarc Workflows Orchestration

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

A financial analytics firm is building an event-driven data ingestion pipeline. Whenever transaction batch files are uploaded to a Cloud Storage bucket, the system must immediately trigger an orchestration process that:

Reads configuration metadata from Cloud Firestore.
Executes a containerized data-cleaning microservice on Cloud Run.
Catches specific HTTP errors (such as 404 Not Found) and retries transient service faults with backoff, while raising unhandled exceptions.

The organization requires a fully serverless, low-latency solution that minimizes operational management and isolates orchestration logic from business logic.

Which architecture should the data engineering team implement to meet these requirements?

Create an Eventarc trigger filtering for google.cloud.storage.object.v1.finalized events that routes directly to a Google Cloud Workflows definition, using built-in Google APIs connectors with try/except and switch blocks in YAML.

Deploy an Apache Airflow DAG in Cloud Composer that uses a GCSObjectsWithPrefixExistenceSensor to poll the bucket, triggering Cloud Run and Firestore using custom Python operators and Airflow retries.

Configure an Application Integration flow using a visual drag-and-drop mapper, setting a Cron schedule trigger to scan the storage bucket and trigger downstream processing.

Deploy a Cloud Run worker pool with minimum instances set to 0, running custom background bash scripts that periodically query the Cloud Storage API and trigger services.

Professional Cloud Data Engineer

Serverless Eventarc Workflows Orchestration

Related questions