Prepare and test your skills

Prepare and test your skills

Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An organization runs an hourly batch data pipeline orchestrated by Cloud Scheduler that invokes an HTTP endpoint hosted on Cloud Run. The service ingests transaction data and writes the processed records into BigQuery.
Due to transient network interruptions, the HTTP target occasionally receives duplicate invocations for a single scheduled run. Furthermore, long-running job attempts sometimes cause Cloud Scheduler to mark an attempt as failed even though the Cloud Run container is still processing the request, blocking subsequent hourly executions.
You need to ensure idempotent execution across retry attempts and resolve the job execution timeout mismatch.
What should you do?
This option relies on the standard HTTP Date header received by the web server to detect duplicate calls and modifies the maxDoublings parameter in the Cloud Scheduler retry policy.
Date header reflects the network transmission timestamp of each individual retry attempt rather than the scheduled execution time. Because each retry occurs at a different time, using Date produces different keys for retries of the same scheduled window, breaking idempotency.maxDoublings only alters the rate at which exponential backoff transitions to linear backoff; it does not adjust attempt_deadline or resolve execution blocking.maxDoublings controls backoff duration progression, not request timeouts.You needed to fine-tune the rate of exponential backoff intervals between failed retries without needing request deduplication.
This solution uses Cloud Scheduler's built-in execution metadata headers to enforce idempotent request handling while tuning the attempt_deadline setting to align Cloud Scheduler's client timeout with the Cloud Run service timeout.
X-CloudScheduler-ScheduleTime header to HTTP requests. This header represents the original scheduled invocation timestamp and remains constant across all automated retry attempts for that specific scheduled interval. Combining the job name with X-CloudScheduler-ScheduleTime produces a deterministic deduplication key, allowing the Cloud Run handler to check if a specific schedule interval has already been processed or is currently executing.attempt_deadline field in Cloud Scheduler ensures Cloud Scheduler waits sufficiently for Cloud Run to respond before timing out the attempt. If attempt_deadline is shorter than the workload processing time, Cloud Scheduler treats the call as failed, initiates retries, or keeps the execution state in AttemptStarted without receiving an AttemptFinished response, which skips subsequent scheduled runs.Cloud Scheduler provides "at-least-once" delivery semantics. Leveraging X-CloudScheduler-ScheduleTime is the standard native mechanism for deduplication because Cloud Scheduler populates this value deterministically for every attempt of a given schedule instance, unlike generic HTTP headers.
This approach has the Cloud Run application query Cloud Logging API in real-time to inspect AttemptStarted and AttemptFinished log entries before executing work, while increasing maxRetryDuration.
maxRetryDuration only controls the maximum cumulative time Cloud Scheduler spends retrying a failed job; it does not extend the single-attempt timeout (attempt_deadline).maxRetryDuration does not prevent premature attempt timeouts.You were building an offline administrative monitoring dashboard to audit job completion history rather than implementing real-time request deduplication.
This approach configures a static payload containing a client-generated UUID in the Cloud Scheduler configuration and sets retryCount to 0 to prevent any failed execution from retrying automatically.
retryCount to 0 disables retries completely, causing transient network glitches to result in permanent data loss or pipeline failures.attempt_deadline, leaving the client-side timeout mismatch unresolved.The workload required strictly "at-most-once" execution where failed jobs must never be retried under any circumstance and the payload schema was completely static.
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
Keep the momentum going with these hand-picked practice scenarios
Last updated Validated