Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Handling Duplicate Eventarc Events Idempotently
Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
Your team is designing an event-driven data pipeline using Eventarc to trigger a Cloud Run microservice whenever a new file is uploaded to Cloud Storage. The microservice processes the file and updates a stateful database.
Because Eventarc guarantees at-least-once delivery, the microservice occasionally receives duplicate events for the same file upload, causing duplicate database entries. You need to implement a solution to prevent data duplication and ensure the database is only updated once per event.
What should you do?
Explanations
Configure the Eventarc trigger to use exactly-once delivery mode to ensure the microservice receives the event only one time
Configure the Eventarc trigger to use exactly-once delivery mode to ensure the microservice receives the event only one time
What exactly-once delivery mode is
Exactly-once delivery is a messaging concept where the infrastructure guarantees that a message is delivered to the consumer one and only one time, completely eliminating duplicates or losses at the network layer.
Why it doesn't meet the requirement
Eventarc does not offer a configuration toggle for an exactly-once delivery mode. It operates strictly on an at-least-once delivery model to prioritize high availability and reliability. Because the infrastructure cannot guarantee exactly-once delivery, the responsibility for deduplication must fall on the receiving consumer.
Key characteristics/limitations
- Eventarc guarantees that an event will be delivered at least once, but makes no guarantees against duplication.
- Network partitions, timeouts, or service restarts can cause the same event to be redelivered.
- Attempting to configure a non-existent exactly-once mode is not possible and will not solve the data duplication issue.
This answer would be correct if
You were using a messaging system that natively supported and required exactly-once delivery configuration at the topic or subscription level (such as specific configurations in Apache Kafka or Pub/Sub exactly-once delivery), rather than Eventarc.
Write a custom polling script inside the microservice to continuously check the Eventarc event router for duplicate messages before processing the payload
Write a custom polling script inside the microservice to continuously check the Eventarc event router for duplicate messages before processing the payload
What custom polling is
Polling is a technique where a client continuously requests data from a server or message broker at regular intervals to check for new or duplicate messages, rather than waiting for the server to push the data.
Why it doesn't meet the requirement
Event-driven architectures using Eventarc rely on push-based messaging. The event router automatically pushes the event to the microservice (e.g., via an HTTP POST request to Cloud Run). There is no endpoint on the Eventarc router for a microservice to poll and check for duplicates.
Key characteristics/limitations
- Eventarc pushes events directly to the configured destination, eliminating the need for polling.
- Polling introduces unnecessary network I/O, latency, and compute overhead.
- Attempting to poll a push-based router breaks the decoupled, asynchronous nature of event-driven architectures.
This answer would be correct if
The architecture was explicitly designed around a pull-based messaging system (like a standard Pub/Sub pull subscription or an SQS queue) where the worker must actively fetch messages and manage their visibility timeouts.
Extract the xgooglemessageuid attribute from the CloudEvent and use it to perform a transactional check in the database to verify if the event has already been processed
Extract the xgooglemessageuid attribute from the CloudEvent and use it to perform a transactional check in the database to verify if the event has already been processed
What xgooglemessageuid is
The xgooglemessageuid is a CloudEvents attribute provided by Eventarc that uniquely identifies the action of publishing an event. It acts as a unique identifier for every distinct event routed through the system.
How it meets the requirement
By extracting this unique identifier from the incoming event payload, the microservice can perform a transactional check against the database (such as querying a table of processed event IDs) before applying the update. If the ID is already present, the service recognizes the event as a duplicate and safely ignores it, effectively achieving exactly-once processing semantics.
Key advantages/capabilities
- Guarantees idempotency: Ensures that processing the same event multiple times yields the exact same state without unintended side effects.
- Safe retries: Complements Eventarc's at-least-once delivery model by making automatic retries completely safe.
- Data integrity: Prevents duplicate records or unintended state mutations in the target database.
- Standardized tracking: Leverages standard CloudEvents attributes, ensuring consistent implementation across different services.
Why this is optimal
Eventarc guarantees at-least-once delivery, meaning duplicate events can and will occur due to network retries or service restarts. The most robust architectural pattern to handle this is designing the receiving service to be idempotent. Using the built-in xgooglemessageuid for deduplication via an external transactional check is the recommended approach for state management in event-driven systems.
Enable the deduplicate_events flag on the Eventarc trigger to automatically filter and drop duplicate events before they are routed to the microservice
Enable the deduplicate_events flag on the Eventarc trigger to automatically filter and drop duplicate events before they are routed to the microservice
What event deduplication flags are
A deduplication flag or setting is a feature found in some messaging brokers that attempts to filter out duplicate messages at the infrastructure level before they ever reach the consumer application.
Why it doesn't meet the requirement
Eventarc does not have a deduplicate_events flag or a built-in filtering mechanism to drop duplicates before routing them to the target service. Eventarc routes events as they occur or as they are retried by the underlying infrastructure.
Key characteristics/limitations
- Eventarc relies entirely on the receiving application to handle idempotency.
- Broker-side deduplication often requires strict time windows and can still fail in distributed edge cases.
- Relying on non-existent infrastructure flags leaves the system vulnerable to data corruption and duplicate entries.
This answer would be correct if
You were using a service that explicitly supported a deduplication configuration flag at the trigger or router level. Because Eventarc does not support this, the application code must handle the deduplication.
Try 10 interactive questions like this one, free
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
- 1,353 practice questions across all 5 exam domains
- Every answer explained: why it is right, and why the others are not
- Scout, an AI tutor that answers your follow-up questions
- Readiness tracking by domain, so you know when to book the exam
Related questions
Keep the momentum going with these hand-picked practice scenarios
AdvancedProfessional Data Engineer
MIG Autohealing vs Load Balancer Health Checks
Designing system for fault tolerance and managing restarts
AdvancedProfessional Data Engineer
Eventarc Asynchronous Event-Driven Architecture
Designing system for fault tolerance and managing restarts
AdvancedProfessional Data Engineer
BigQuery GoogleSQL UDFs With MAX_BY
Processing logic
AdvancedProfessional Data Engineer
Google Cloud Resource Hierarchy for Data Governance
Designing the project, dataset, and table architecture to ensure proper data governance
AdvancedProfessional Data Engineer
Cloud Spanner INSERT_OR_UPDATE Idempotent Writes
Defining data transformation and orchestration logic
IntermediateProfessional Cloud Architect
Google Cloud Migration Center TCO
Success measurements (e.g., key performance indicators [KPI], return on investment [ROI], and metrics)
Last updated Validated