Prepare and test your skills
Prepare and test your skills
Cloud Pub/Sub (opens in a new tab) is a messaging service that decouples publishers from subscribers through topics and subscriptions. A publisher sends each message to a topic, and Pub/Sub delivers a copy of each message to every attached subscription for independent processing. This separation lets services scale, fail, and deploy independently while Pub/Sub handles delivery, retention, and replay.
A pull subscription is a delivery mode where the subscriber client requests messages from the Pub/Sub server. Any device on the internet with authorized credentials can call the Pub/Sub API, so the subscriber does not need a public inbound endpoint. Multiple subscriber clients can pull from the same subscription at the same time, and each client receives a subset of messages for parallel processing.
Pull delivery gives flow control to the subscriber. The client controls the delivery rate and can extend the acknowledgment deadline, which allows processing to take an arbitrarily long time. Pull achieves high throughput at low CPU and bandwidth through batched delivery, batched acknowledgments, and massively parallel consumption, but frequent polling only to reduce delivery time can be inefficient. For general use, rely on the high-level client library and avoid setting the deprecated returnImmediately field to true because it harms pull performance.
A push subscription is a delivery mode where the Pub/Sub server initiates delivery to a subscriber application. Data flows from the Pub/Sub server to an HTTPS endpoint that must be reachable on the public web with a DNS name and a non-self-signed SSL certificate. The receiving endpoint can be decoupled from the subscription, so messages from multiple subscriptions can be sent to one webhook endpoint. Push delivery gives flow control to the server. The Pub/Sub server automatically controls the rate and delivers one message per request up to a limit on outstanding messages, so clients do not manage message flow themselves. A client that cannot handle the current load signals backpressure by returning an HTTP error, and push endpoints can sit behind load balancers to spread work. Choose push when the subscriber cannot make outgoing requests, cannot include the client library, needs one webhook to process multiple topics without knowing the subscription list, or runs on App Engine Standard or Cloud Run functions.
An export subscription is a delivery mode that writes messages directly to a Google Cloud resource without a subscriber client or Dataflow job. The three export types are BigQuery subscriptions that export to a BigQuery table, Cloud Storage subscriptions that export to a Cloud Storage bucket, and Bigtable subscriptions that export to a Bigtable table. Data flows from the Pub/Sub service to the destination resource, and the service automatically balances load, applies flow control, and scales to very high volume. Export subscriptions reduce cost and operations when messages need storage without extra processing:
The destination resource must already exist with appropriate permissions. If the application needs transformations beyond Single Message Transforms (SMTs), use a Dataflow pipeline instead of an export subscription.
Choosing a delivery mode means matching who initiates delivery, who owns the endpoint, and who controls flow to the subscriber design. Pull needs no endpoint configuration, push needs a DNS-reachable HTTPS endpoint except for same-project App Engine apps, and export needs the destination resource and permissions in place.
| Dimension | Pull subscription | Push subscription | Export subscription |
|---|---|---|---|
| Initiator | Subscriber client requests messages from Pub/Sub server | Pub/Sub server sends requests to subscriber HTTPS endpoint | Pub/Sub service writes to Google Cloud resource |
| Endpoint | Any authorized caller of Pub/Sub API, no public endpoint needed | Public HTTPS server with non-self-signed certificate, can be shared or load-balanced | Existing BigQuery table, Cloud Storage bucket, or Bigtable table |
| Flow control | Subscriber controls rate and extends acknowledgment deadline | Server controls rate, client returns HTTP error when overloaded | Service controls rate to optimize writes |
| Throughput | High throughput from batched delivery and parallel pull | One message per request with limit on outstanding messages | Dynamically scaled by Pub/Sub service |
Exam tip: Choose pull when the subscriber can use credentials and the client library and needs high-throughput batched control, and choose push when the subscriber needs Pub/Sub to call a public HTTPS webhook without client-library dependencies.
Reliable processing in Cloud Pub/Sub starts with acknowledging only after successful processing. By default, Pub/Sub discards a message from a subscription after the message is acknowledged, and a message acknowledged before processing is not redelivered if processing later fails. The exception is when acknowledged-message retention or topic retention is configured and an operator performs a seek operation to replay messages.
Subscriber flow control protects subscribers from transient traffic spikes. The client sets maximum outstanding messages and total outstanding message bytes, which define the maximum count and total size of delivered messages without acknowledgments or negative acknowledgments. If either limit is crossed, the subscriber client does not pull more messages, which allows time for autoscaling to respond and saves latency or spreads load over time to save costs. Subscribers with high-latency processing may also need custom flow-control values and lease management for acknowledgment deadlines.
Ordering, dead-letter handling, and exactly-once delivery are reliability features that keep communication predictable between decoupled services. Cloud Pub/Sub supports ordered delivery where publishers publish with ordering keys and subscribers follow ordered-messaging practices, so related messages keep their intended sequence through the same subscription. A subscription can configure a DeadLetterPolicy that forwards undeliverable messages to a dead-letter topic after repeated delivery failures, which isolates bad messages instead of blocking the subscription. A subscription can also enable exactly-once delivery to strengthen processing guarantees when duplicates would cause incorrect results.
These features combine with acknowledgment discipline and flow control. Ordering preserves sequence, dead-letter topics preserve progress by moving aside messages that cannot be processed, and exactly-once delivery reduces duplicate side effects in downstream systems. Together they let independent services communicate through topics without direct calls while keeping delivery predictable under failure and scale.
Eventarc (opens in a new tab) is the Google Cloud event-routing service that lets Application Integration run integration flows in response to events. An Eventarc trigger sits between an event source and an integration flow, so data flows from the source to Eventarc Standard and then to the integration. This pattern decouples producers from consumers because the source publishes an event once and the integration runs only when a matching event arrives.
An Eventarc trigger routes events from Google Cloud services, custom applications, and SaaS providers to an Application Integration flow. The event source emits a change, Eventarc Standard matches it against the trigger filters, and the trigger invokes the connected integration as the destination. Supported sources include Google services such as Pub/Sub, Cloud Storage, and Firestore, plus third-party providers that offer an Eventarc source and developer-defined custom sources. The destination in this pattern is a specific Application Integration flow, which must be connected to the trigger element and published before it can receive events.
Prerequisites and trigger identity define what must exist before an Eventarc trigger for Application Integration can invoke its destination. Eventarc triggers for Application Integration require the Eventarc API and the Pub/Sub API to be enabled in the Google Cloud project. Enabling APIs requires the serviceusage.services.enable permission, which is included in the Owner role (roles/owner) or the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Every trigger also needs an Identity and Access Management (IAM) service account that Eventarc uses as its identity when invoking the integration. If no account is selected, the trigger uses the Compute Engine default service account, and that account must have permission to invoke the destination service.
Trigger types and core settings define which events start the integration and how the event payload is delivered. The first choice is the trigger type, which determines the kind of event provider:
After the type is selected, the trigger needs an event provider and an event type, since the available event types change with the provider. Optional filters narrow matching further, for example to fire only when a specific API resource is updated. The remaining settings control delivery: region, event data content type of application/json or application/protobuf, service account, retry on failure, and network attachment for private delivery. A network attachment connects the service producer Virtual Private Cloud (VPC) network to the consumer VPC network, so Eventarc can reach an internal HTTP endpoint inside the consumer VPC. For best performance, the Eventarc trigger, the monitored Google Cloud resource, and the destination should all be in the same Google Cloud region.
Publishing and trigger lifecycle describe how an Eventarc trigger moves from draft rules to active event routing. Saving the trigger stores the rules for when and how Eventarc responds as part of the integration draft, and the trigger stays inactive until the integration is published. After publication the trigger becomes active and begins processing matching events, so the integration flow depends on publication to receive traffic. Trigger versions follow the integration versions that contain them. If a new integration version is published with a modified trigger configuration, Eventarc automatically replaces the old trigger with a new one and deletes the obsolete trigger. If a new version is published with the same trigger and configuration, no new trigger is created and the existing trigger is reused. If trigger creation fails during publication because of configuration errors, the integration still publishes but the trigger stays inactive, so the fix is to unpublish the integration, correct the trigger, and republish while checking status in the console notification center.
Exam tip: Saving an Eventarc trigger only stores its definition; you must publish the integration to deploy the rules and make the trigger active.
Choosing a creation path means choosing where the Eventarc trigger definition starts, while both paths end in the same published integration flow. Creation from Application Integration starts inside the integration editor, while creation from Eventarc starts on the Triggers page and then opens the integration.
| Creation path | Starting point | How destination is set |
|---|---|---|
| Application Integration | Add the Eventarc element from Triggers in the integration editor, then open Configure Eventarc trigger | The trigger is already attached to the open integration; connect it to the flow and publish |
| Eventarc console | Create trigger on the Eventarc Triggers page and select Application Integration as event destination | Select the target integration, which opens as a new draft if already published, then verify, connect, and publish |
Both paths require the same core inputs of trigger name, type, provider, event type, region, and service account, and both require publication before events flow to the integration.
Ensuring reliability and idempotency in asynchronous systems means transient failures do not lose work and repeated deliveries do not corrupt data. Reliability comes from robust retry policies that automatically try a failed operation again with controlled delays and timeouts. Idempotency means processing logic produces the same final result even if the same event is delivered more than once during retries or high-throughput spikes.
Configuring retry and timeout policies for transient failures controls how an application calls AlloyDB for PostgreSQL when retrieving cluster information with a getCluster request. The policy is built with AlloyDBAdminSettings (opens in a new tab) and fine-tunes how long the client waits, how fast delays grow, and when the client stops trying. Because transient errors such as brief network or service unavailability often succeed on a later attempt, bounded retries improve resilience without waiting forever.
The getCluster retry policy tunes eight related values that work together as backoff and timeout bounds:
The retry delay starts small and then grows by the multiplier until it hits the maximum, while the per-attempt timeout grows separately toward its own maximum. The attempt count and total timeout act as outer boundaries, so retries stop when either 5 attempts are used or 300 seconds have passed. This combination lets throughput stay high during event spikes because short failures are retried quickly, long failures back off, and stuck operations still give up within a known budget.
Controlling retries in event-driven functions determines whether the platform delivers the same event again after a function runs. In a flow where data moves from Pub/Sub through Eventarc to Cloud Run functions (opens in a new tab), the function signals success or failure by how it exits. Throwing an exception causes the execution to be treated as a failure, while returning normally ends processing of that event. The function chooses between two exit behaviors for each event:
| Exit behavior | Platform result | Choose when |
|---|---|---|
| Throw an exception | Platform returns an error and the original invocation is re-sent for retry | The failure is intermittent and another attempt might succeed |
| Return normally | Platform ends processing and the event is not re-sent | The work completed or the failure is non-retriable and retry would not help |
The decision is often driven by event data, such as a retry flag or some_parameter in the Pub/Sub message payload. The function reads the payload, attempts its work, and then throws to request redelivery or returns to stop. Because a throw re-sends the same invocation, one logical event can arrive at the function multiple times.
Exam tip: Throw an exception to trigger a retry; return normally to stop processing — both end the current execution, but only a throw causes the event to be delivered again.
Idempotent processing under duplicate delivery ensures that handling the same event twice leaves data in the same consistent state as handling it once. Idempotency is required because retry policies re-send the original invocation after intermittent failures, so at-least-once delivery is the normal behavior during spikes and outages. A function that updates a database, sends a message, or changes cluster state must therefore tolerate repeats without creating duplicates or partial updates. Robust logic separates transient failures from permanent failures before deciding to retry. Retry only when the error is intermittent, such as a timeout reading cluster information that might succeed on the next attempt with backoff. Return normally without throwing when the failure is non-retriable, so a bad payload or business-rule violation does not loop through all retry attempts and consume the total timeout budget. This discipline keeps distributed data consistent while still allowing automatic recovery from short-lived faults.
Use pull when the subscriber can use credentials and the client library and needs high-throughput batched control, use push when the subscriber needs Cloud Pub/Sub to call a public HTTPS webhook without client-library dependencies, and use export when messages need storage without extra processing. Pull delivery is initiated by the subscriber, push delivery is initiated by the Cloud Pub/Sub server to a public HTTPS endpoint, and export delivery is written directly to a BigQuery table, Cloud Storage bucket, or Bigtable table.
The Eventarc API and the Pub/Sub API must be enabled in the Google Cloud project, which requires permission included in the Owner role or the Service Usage Admin role. Every trigger also needs an Identity and Access Management service account that Eventarc uses as its identity when invoking the integration, and if no account is selected the trigger uses the Compute Engine default service account which must have permission to invoke the destination service.
A Cloud Run function requests retry by throwing an exception, which causes the platform to return an error and re-send the original invocation, and it stops redelivery by returning normally, which ends processing of that event. Throw when the failure is intermittent and another attempt might succeed, and return normally when the work completed or the failure is non-retriable, while keeping processing idempotent because the same event can arrive multiple times.
Last updated