Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Google Cloud feature
Log sinks
Topic in 1 exam domain
A routing mechanism in Cloud Logging that controls how log entries are exported.
Where this appears on the exam
1 objective in 1 of 5 domains
Domain 3
Storing the data~20% of exam
Related topics
FoundationsBuilding blocksThis topicWorks alongsideApplication Default CredentialsFeaturecustomer-managed encryption keysFeatureDataflowServicePub/SubServiceGoogle Cloud StorageServiceCloud StorageServiceCloud LoggingServiceCloud Run functionsServiceBigQueryServiceCloud RunService
Log sinksFeature
Google SecOpsServicePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- Log sinks works with Pub/Sub From the official documentation: “For logging data, Logging provides exports (log sinks) to [Pub/Sub](/pubsub).”
- BigQuery works with Log sinks From the official documentation: “You can configure your log sinks so that log data isn't stored in log buckets but is instead routed to one of the following destinations: - [Cloud Storage pricing](https://cloud.google.com/storage/pricing). - [BigQuery pricing](https://cloud.google.com/bigquery/pricing).”
- Cloud Storage works with Log sinks From the official documentation: “You can configure your log sinks so that log data isn't stored in log buckets but is instead routed to one of the following destinations: - [Cloud Storage pricing](https://cloud.google.com/storage/pricing).”
- Google Cloud Storage works with Log sinks From the official documentation: “Cloud Logging uses log sinks to route log entries to supported destinations, including Cloud Storage buckets.”
- Log sinks is part of Cloud Logging From the official documentation: “For logging data, Logging provides exports (log sinks) to [Pub/Sub](/pubsub).”
- Application Default Credentials works with Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials secures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- BigQuery depends on Application Default Credentials From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- Dataflow depends on Application Default Credentials From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials works with Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
- BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
- Application Default Credentials works with BigQuery From the official documentation: “Create a BigQuery client using application default credentials.”
- Application Default Credentials secures Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Pub/Sub depends on Application Default Credentials From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Cloud Storage works with Google SecOps From the official documentation: “Configure a feed in Google SecOps to ingest from Google Cloud Storage”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- Application Default Credentials works with Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials secures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials works with Google Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Cloud Logging works with customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
- Cloud Run works with Pub/Sub From the official documentation: “Route Cloud Pub/Sub events to Cloud Run”
- Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
- Cloud Run works with Cloud Storage From the official documentation: “Route Cloud Storage events to Cloud Run”
- Google Cloud Storage works with Google SecOps From the official documentation: “Google SecOps uses a unique service account to read data from your GCS bucket.”
- Application Default Credentials works with Cloud Run functions From the official documentation: “To authenticate to Cloud Run functions, set up Application Default Credentials.”
- Cloud Run functions depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Run functions, set up Application Default Credentials.”
- BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- Cloud Run functions works with Cloud Storage From the official documentation: “Set up functions in response to events in Cloud Storage:”
- BigQuery works with Cloud Run functions From the official documentation: “BigQuery supports external function calls through Cloud Run functions.”
- BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
- Cloud Run depends on Pub/Sub From the official documentation: “Create a Pub/Sub topic that Cloud Scheduler will publish to and the Cloud Run function will subscribe to.”
- Cloud Run functions works with Pub/Sub From the official documentation: “Use Cloud Run functions to publish messages on Pub/Sub.”
- Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
- customer-managed encryption keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Cloud Run depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Run, set up Application Default Credentials.”
- Cloud Run functions depends on Pub/Sub From the official documentation: “The Pub/Sub message triggers Cloud Run functions.”
- customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
- customer-managed encryption keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
- Application Default Credentials secures Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Cloud Storage works with customer-managed encryption keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- Application Default Credentials works with Cloud Run From the official documentation: “To authenticate to Cloud Run, set up Application Default Credentials.”
- Application Default Credentials secures Cloud Run functions From the official documentation: “To authenticate to Cloud Run functions, set up Application Default Credentials.”
- Cloud Logging depends on Application Default Credentials From the official documentation: “To authenticate to Logging, set up Application Default Credentials.”
- Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
- Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
- Cloud Run is an alternative to Cloud Run functions From the official documentation: “To create remote functions, you must [grant required roles](/bigquery/docs/remote-functions#grant_permission_on_function) to Cloud Run functions or Cloud Run.”
- BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
- BigQuery works with Google SecOps From the official documentation: “Google SecOps exports the following categories of data to your BigQuery project:”
- Cloud Run works with Google Cloud Storage From the official documentation: “The Cloud Run function will receive syslog messages from FortiEDR via HTTP and write them to GCS.”
- Cloud Run functions works with Google Cloud Storage From the official documentation: “Service account for Dataminr Alerts Cloud Run function to write alert data to GCS”
- Google Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- BigQuery works with Cloud Run From the official documentation: “You can then use Cloud Run to transcode data present in the bucket and move it to BigQuery.”
- Cloud Logging works with Cloud Run From the official documentation: “Cloud Logging captures, stores, and analyzes Cloud Run logs.”
How learners score on this topic
Last updated