A retail company is designing an event-driven architecture to process user interactions across its e-commerce website in real time. The solution must satisfy the following technical requirements: Ingest high-throughput, asynchronous event streams from web clients globally with sub-second latency Perform low-latency transformations, event deduplication, and complex windowed aggregations (such as 15-minute sliding session metrics) Stream the enriched, analytical results directly into BigQuery for near-real-time dashboarding Provide a fully managed, auto-scaling operational model without requiring manual cluster management Which architecture should the company implement?

Ingest user events with Cloud Pub/Sub and process the stream using a Dataflow pipeline that outputs to BigQuery: ### What Cloud Pub/Sub and Dataflow Provide **Cloud Pub/Sub** is a globally distributed, fully managed, asynchronous messaging service designed to ingest and distribute high-throughput event streams with latencies typically around 100 milliseconds. **Dataflow** is a serverless, managed stream and batch data processing service built on **Apache Beam**, providing native support for complex transformations, stateful windowing, and streaming data sinks. ### How It Meets the Requirements - **High-throughput Ingestion**: Cloud Pub/Sub ingests events asynchronously across global endpoints, scaling dynamically without provisioning servers. - **Windowing and Transformations**: Dataflow natively supports advanced stream processing primitives, including sliding windows, session analysis, watermarks, and built-in **message deduplication**. - **Direct BigQuery Ingestion**: Dataflow provides native streaming connectors to insert enriched records directly into **BigQuery** tables for near-real-time analytics. - **Fully Managed Autoscaling**: Both services are completely serverless, removing the operational overhead of cluster provisioning, capacity planning, and manual scaling. ### Key Advantages and Capabilities - **At-Least-Once Delivery & Deduplication**: Pub/Sub provides durable at-least-once delivery, while Dataflow applies deterministic record deduplication to avoid double counting. - **Rich Windowing Semantics**: Dataflow enables developers to express fixed, sliding, and session windows over streaming data with ease. - **Unified Programming Model**: Apache Beam pipelines allow the reuse of business logic across both streaming and batch historical analyses. - **Zero Infrastructure Management**: Google Cloud automatically provisions, scales, and optimizes the compute workers executing the Dataflow job. ### Why This Is Optimal Combining **Cloud Pub/Sub** with **Dataflow** represents Google Cloud's premier reference architecture for real-time streaming analytics. It delivers end-to-end low latency, robust state management for time-based metrics, and elastic scalability without infrastructure maintenance.

StudyCerts

Realtime Event Processing Pubsub Dataflow Bigquery | Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam! | StudyCerts

Professional Cloud Architect

Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam!

Intermediate

Prepare and test your skills

Track Readiness

Set Exam Goal

Intermediate

Prepare and test your skills

Track Readiness

Set Exam Goal

Realtime Event Processing Pubsub Dataflow Bigquery

Section: Choosing data processing solutions

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

A retail company is designing an event-driven architecture to process user interactions across its e-commerce website in real time. The solution must satisfy the following technical requirements:

Ingest high-throughput, asynchronous event streams from web clients globally with sub-second latency
Perform low-latency transformations, event deduplication, and complex windowed aggregations (such as 15-minute sliding session metrics)
Stream the enriched, analytical results directly into BigQuery for near-real-time dashboarding
Provide a fully managed, auto-scaling operational model without requiring manual cluster management

Which architecture should the company implement?

Ingest user events with Cloud Pub/Sub and invoke Cloud Functions (2nd gen) to execute windowed aggregations and persist records into BigQuery

Upload events into Cloud Storage buckets and orchestrate hourly Dataproc Spark batch jobs using Cloud Composer to load BigQuery

Ingest user events with Cloud Pub/Sub and process the stream using a Dataflow pipeline that outputs to BigQuery

Ingest user events into Compute Engine instances managed by an instance group, buffer data on local Persistent Disks, and run cron jobs to insert records into BigQuery

Professional Cloud Architect

Realtime Event Processing Pubsub Dataflow Bigquery

Related questions