Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Google Cloud service
Pub/Sub
Topic in 44 exam domains
Also called: Cloud Pub/Sub, Cloud PubSub, GCP Pub/Sub, Google Cloud Pub/Sub, Google Cloud PubSub, Google Pub/Sub, Pub/Sub API, Pub/Sub service, PubSub
A messaging service used to trigger asynchronous processing workflows and invoke functions.
Where this appears on the exam
32 objectives in 5 of 5 domains
~22% of exam
~25% of exam
- Data cleansing
- Identifying the services (e.g., Dataflow, Apache Beam, Dataproc, Cloud Data Fusion, BigQuery, Pub/Sub, Apache Spark, Hadoop ecosystem, and Apache Kafka)
- Transformations
- Batch
- Streaming (e.g., windowing, late arriving data)
- Processing logic
- AI data enrichment
- Data acquisition and import
- Integrating with new data sources
Domain 3
Storing the data~20% of exam
~15% of exam
~18% of exam
Related topics
FoundationsThis topicWorks alongsideAppliedVPC Service ControlsServiceApplication Default CredentialsFeatureCMEKFeaturecustomer-managed encryption keysFeatureIdentity and Access ManagementServiceBigQueryServiceCloud StorageServiceCompute EngineServiceDataflowService
Pub/SubService
Secret ManagerServiceSensitive Data ProtectionServiceCloud SQLServiceGKEServiceWorkflowsServiceCloud RunServiceCloud Run functionsServiceEventarcServiceCloud SchedulerServicePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- Application Default Credentials works with Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Application Default Credentials secures Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Pub/Sub depends on Application Default Credentials From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Cloud Run works with Pub/Sub From the official documentation: “Route Cloud Pub/Sub events to Cloud Run”
- Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
- Pub/Sub works with Secret Manager From the official documentation: “Receive notifications via Pub/Sub when Secret Manager resources change.”
- Pub/Sub works with Sensitive Data Protection From the official documentation: “Watch the Pub/Sub topic until the DLP job finishes”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- CMEK secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
- Cloud Run depends on Pub/Sub From the official documentation: “Create a Pub/Sub topic that Cloud Scheduler will publish to and the Cloud Run function will subscribe to.”
- Cloud Run functions works with Pub/Sub From the official documentation: “Use Cloud Run functions to publish messages on Pub/Sub.”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Cloud Run functions depends on Pub/Sub From the official documentation: “The Pub/Sub message triggers Cloud Run functions.”
- customer-managed encryption keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- Cloud Scheduler works with Pub/Sub From the official documentation: “Create a Pub/Sub topic to set up as your Cloud Scheduler job target.”
- Eventarc works with Pub/Sub From the official documentation: “Trigger functions from Pub/Sub using Eventarc”
- GKE works with Pub/Sub From the official documentation: “Route Cloud Pub/Sub events to GKE”
- Pub/Sub works with Workflows From the official documentation: “Route Cloud Pub/Sub events to Workflows”
- Eventarc depends on Pub/Sub From the official documentation: “Eventarc uses Pub/Sub as its transport layer and has a default retry policy that might not work well for your function.”
- Identity and Access Management secures Pub/Sub From the official documentation: “Pub/Sub uses Identity and Access Management (IAM) for authorization.”
- Cloud Scheduler depends on Pub/Sub From the official documentation: “Cloud Pub/Sub (required for Cloud Scheduler to invoke functions)”
- CMEK works with Pub/Sub From the official documentation: “By using Pub/Sub with CMEK, you are able to meet locality rules about encryption while still maintaining high availability.”
- Eventarc is an alternative to Pub/Sub From the official documentation: “Instead of using Eventarc, you can use Pub/Sub to directly push messages to your service.”
- Pub/Sub depends on Cloud Run functions From the official documentation: “After you have set up Cloud Run functions to process model output and generate events, you can set up event notification with Pub/Sub.”
- Application Default Credentials works with Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Sensitive Data Protection depends on Application Default Credentials From the official documentation: “To authenticate to Sensitive Data Protection, set up Application Default Credentials.”
- Application Default Credentials secures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- BigQuery depends on Application Default Credentials From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- BigQuery works with Sensitive Data Protection From the official documentation: “Uses the Data Loss Prevention API to analyze BigQuery data.”
- Dataflow depends on Application Default Credentials From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
- Application Default Credentials works with BigQuery From the official documentation: “Create a BigQuery client using application default credentials.”
- Application Default Credentials secures Sensitive Data Protection From the official documentation: “To authenticate to Sensitive Data Protection, set up Application Default Credentials.”
- Application Default Credentials works with Secret Manager From the official documentation: “To authenticate to Secret Manager, set up Application Default Credentials.”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
- Cloud Run works with Cloud SQL From the official documentation: “Learn how to connect to Cloud SQL from Cloud Run”
- Application Default Credentials works with Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials works with Compute Engine From the official documentation: “To authenticate to Compute Engine, set up Application Default Credentials.”
- VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
- Application Default Credentials secures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Cloud SQL works with Cloud Storage From the official documentation: “When ingesting data from Cloud SQL, data is first staged to a Cloud Storage bucket.”
- Cloud Run works with Cloud Scheduler From the official documentation: “Create a Cloud Scheduler job that triggers the Cloud Run service daily.”
- Cloud SQL works with Compute Engine From the official documentation: “Create a Compute Engine VM Instance to host a sample web app that connects to Cloud SQL.”
- Cloud Run works with Eventarc From the official documentation: “Route Eventarc events to Cloud Run”
- Cloud Run works with Cloud Storage From the official documentation: “Route Cloud Storage events to Cloud Run”
- Cloud Run works with Secret Manager From the official documentation: “You can integrate Secret Manager with other Google Cloud products such as Cloud Run.”
- Application Default Credentials works with Cloud Run functions From the official documentation: “To authenticate to Cloud Run functions, set up Application Default Credentials.”
- Cloud Run functions depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Run functions, set up Application Default Credentials.”
- Application Default Credentials secures Compute Engine From the official documentation: “To authenticate to Compute Engine, set up Application Default Credentials.”
- Application Default Credentials works with Sensitive Data Protection From the official documentation: “To authenticate to Sensitive Data Protection, set up Application Default Credentials.”
- Cloud Run functions works with Cloud Scheduler From the official documentation: “Create a Cloud Scheduler job to trigger the Cloud Run function on a regular schedule.”
- Cloud Run functions works with Cloud Storage From the official documentation: “Set up functions in response to events in Cloud Storage:”
- Compute Engine is an alternative to GKE From the official documentation: “To use TPU7x, you can use Google Kubernetes Engine (GKE) or Compute Engine.”
- Secret Manager depends on Application Default Credentials From the official documentation: “To authenticate to Secret Manager, set up Application Default Credentials.”
- BigQuery works with Cloud Run functions From the official documentation: “BigQuery supports external function calls through Cloud Run functions.”
How learners score on this topic
Last updated