Streaming Dataflow Spanner Idempotent Sink

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An ecommerce company runs an Apache Beam streaming pipeline on Google Cloud Dataflow to track product inventory updates from high-throughput retail transactions. The pipeline performs stateful aggregations using Combine.perKey within fixed temporal windows configured with early and late triggers to handle incremental updates and late-arriving data.

The resulting inventory totals must be written to Cloud Spanner. Because streaming Dataflow pipelines retry failed worker tasks indefinitely, the write mechanism must be fully idempotent, prevent primary key collision errors during retries or multiple trigger firings for the same window key, and maintain exactly-once sink semantics without pipeline stalls.

Which approach should you implement to write the aggregated inventory totals to Cloud Spanner?

Apply a custom ParDo transform that executes direct DML INSERT statements using the Cloud Spanner JDBC driver inside transactional boundaries.

Sink the aggregated records using SpannerIO.write() configured with explicit DELETE followed by INSERT mutations in separate sequential pipeline branches.

Construct INSERT_OR_UPDATE mutations for the aggregated records and commit them using SpannerIO.write().

Construct INSERT mutations for the aggregated records and commit them using SpannerIO.write().

Professional Cloud Data Engineer

Streaming Dataflow Spanner Idempotent Sink

Related questions