Cloud Scheduler Idempotency Deduplication Timeout Configuration

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An organization runs an hourly batch data pipeline orchestrated by Cloud Scheduler that invokes an HTTP endpoint hosted on Cloud Run. The service ingests transaction data and writes the processed records into BigQuery.

Due to transient network interruptions, the HTTP target occasionally receives duplicate invocations for a single scheduled run. Furthermore, long-running job attempts sometimes cause Cloud Scheduler to mark an attempt as failed even though the Cloud Run container is still processing the request, blocking subsequent hourly executions.

You need to ensure idempotent execution across retry attempts and resolve the job execution timeout mismatch.

What should you do?

Query Cloud Logging from the Cloud Run container for AttemptStarted log entries to detect duplicate executions, and increase maxRetryDuration in Cloud Scheduler.

Extract the job name and the X-CloudScheduler-ScheduleTime header in the Cloud Run handler to implement request deduplication, and configure the attempt_deadline field in Cloud Scheduler to align with the container timeout.

Configure Cloud Scheduler to use the standard HTTP Date header for deduplication, and adjust the maxDoublings setting to reduce retry intervals.

Generate a unique UUID inside the Cloud Scheduler request payload body and disable retries by setting retryCount to 0, while increasing Cloud Run's concurrency limit.

Professional Cloud Data Engineer

Cloud Scheduler Idempotency Deduplication Timeout Configuration

Related questions