An e-commerce company is designing an event-driven inventory management pipeline on Google Cloud. The system ingests high-volume inventory mutation events from physical stores and online checkouts through Pub/Sub and processes them using Cloud Dataflow before updating product inventory levels in BigQuery and downstream datastores. The system must meet the following reliability and data integrity requirements: Guarantee inventory count accuracy and consistency despite potential out-of-order and duplicate message delivery caused by transient network retries and at-least-once Pub/Sub delivery. Prevent invalid or malformed messages from stalling the streaming pipeline while preserving them for debugging and compensating corrections. Allow safe recovery and message reprocessing in the event of subscriber-side processing bugs. Which architectural pattern should the data engineering team implement?

Publish state-delta events (e.g., relative inventory adjustments) with unique event IDs, use Dataflow for deduplication and stream processing, configure Pub/Sub dead-letter topics for failed deliveries, and enable message retention with snapshots on subscriptions for replay recovery.: ### What this architectural pattern is This architecture combines **event-driven state-delta payloads**, **Dataflow deduplication**, **Pub/Sub dead-letter topics**, and **subscription snapshots/seek** to achieve end-to-end transactional integrity, commutativity, and fault tolerance across a streaming ingestion pipeline. ### How it meets all requirements - **Consistency & Duplicate Handling**: Publishing commutative state deltas (such as `{"inventory": -1}`) rather than absolute state overwrite commands ensures that commutative state modifications remain mathematically consistent regardless of message arrival sequence. Applying **Dataflow deduplication** filters out duplicate deliveries inherent to Pub/Sub's at-least-once model using distinct message identifiers or record keys. - **Error Handling & Fault Isolation**: Configuring **dead-letter topics** ensures that invalid or unprocessable messages (poison pills) are isolated after exceeding retry thresholds without blocking the streaming pipeline workers. - **Disaster Recovery & Replay**: Enabling subscription **message retention** and capturing **snapshots** before pipeline deployments allows the engineering team to replay unacknowledged and acknowledged messages if downstream application defects or unexpected data mutations occur. ### Key advantages and capabilities - **Commutative state operations**: State-delta events eliminate strict FIFO sequencing dependencies and make rollbacks or compensating transactions straightforward. - **Exactly-once processing semantics**: Dataflow natively integrates with Pub/Sub and destination sinks (like BigQuery) to enforce stateful, exactly-once record processing. - **Automated poison-pill handling**: Dead-letter queues protect processing capacity from crashing on malformed payloads. - **Point-in-time recovery**: Subscription snapshots and seek operations provide a safety net for operational rollbacks. ### Why this is optimal This pattern balances high-throughput asynchronous scaling with strict data fidelity and ACID-like state consistency. It eliminates single-point-of-failure sequential bottlenecks while guaranteeing durable, idempotent data handling.

Event Driven Inventory Transactional Integrity Pubsub Dataflow | professional-cloud-data-engineer

Event Driven Inventory Transactional Integrity Pubsub Dataflow

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An e-commerce company is designing an event-driven inventory management pipeline on Google Cloud. The system ingests high-volume inventory mutation events from physical stores and online checkouts through Pub/Sub and processes them using Cloud Dataflow before updating product inventory levels in BigQuery and downstream datastores.

The system must meet the following reliability and data integrity requirements:

Guarantee inventory count accuracy and consistency despite potential out-of-order and duplicate message delivery caused by transient network retries and at-least-once Pub/Sub delivery.
Prevent invalid or malformed messages from stalling the streaming pipeline while preserving them for debugging and compensating corrections.
Allow safe recovery and message reprocessing in the event of subscriber-side processing bugs.

Which architectural pattern should the data engineering team implement?

Publish state-delta events (e.g., relative inventory adjustments) with unique event IDs, use Dataflow for deduplication and stream processing, configure Pub/Sub dead-letter topics for failed deliveries, and enable message retention with snapshots on subscriptions for replay recovery.

Use Pub/Sub standard pull subscriptions directly from Compute Engine virtual machines, set the acknowledgment deadline to zero seconds, and rely on BigQuery table snapshots for message deduplication.

Publish absolute inventory state commands (e.g., setInventoryLevel) with ordering keys enabled in Pub/Sub, and have Dataflow process messages sequentially using a single regional worker node without dead-letter topics.

Configure Pub/Sub push subscriptions to invoke Cloud Functions that write raw messages directly to Cloud Storage cold archive buckets, triggering hourly Dataproc batch jobs for reconciliation.

Professional Cloud Data Engineer

Event Driven Inventory Transactional Integrity Pubsub Dataflow

Related questions