pipelines

Related topics

• pipelines is part of BigQuery From the official documentation: “You can create [pipelines in BigQuery](/bigquery/docs/pipelines-introduction), which let you orchestrate and automate workflows such as scheduling SQL queries, notebooks, and data preparation tasks to run in sequence.”
• pipelines is part of Eventarc From the official documentation: “Manage Eventarc pipelines.”
• VPC Service Controls secures pipelines From the official documentation: “If you use VPC Service Controls to protect your pipelines, you should be aware that pipeline runs are powered by Dataform.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Storage works with Eventarc From the official documentation: “Trigger functions from Cloud Storage using Eventarc”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Audit Logs works with Cloud Storage From the official documentation: “This page provides supplemental information for using [Cloud Audit Logs](/logging/docs/audit) with Cloud Storage.”
• Cloud Audit Logs works with VPC Service Controls From the official documentation: “VPC Service Controls integrate with Cloud Audit Logs to monitor access to the control plane.”
• Cloud Logging works with Cloud Monitoring From the official documentation: “You can store and view query-response activity logs in Cloud Logging and set up logs-based monitoring by using Cloud Monitoring.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• Eventarc works with Pub/Sub From the official documentation: “Trigger functions from Pub/Sub using Eventarc”
• BigQuery works with Cloud Audit Logs From the official documentation: “Cloud Audit Logs maintain a record of Google Cloud events, including BigQuery events.”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Cloud Audit Logs works with Pub/Sub From the official documentation: “Working with audit logs Learn about the default and optional logs created by Pub/Sub as part of Cloud Audit Logs”
• Cloud Logging is an alternative to Cloud Storage From the official documentation: “Instead, they are routed to either Cloud Storage (recommended) or Cloud Logging, depending on your organization's configuration.”
• Cloud Logging works with Eventarc From the official documentation: “Eventarc automatically sends audit logs to Cloud Logging for troubleshooting and streams metrics to Cloud Monitoring for trend analysis and alerting.”
• Cloud Monitoring works with Dataflow From the official documentation: “Dataflow is fully integrated with Cloud Monitoring.”
• Cloud Monitoring works with Pub/Sub From the official documentation: “Use Cloud Monitoring and Pub/Sub to set up custom monitoring and alerting policies for exported logs.”
• Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
• Dataflow works with Eventarc From the official documentation: “Dataflow integration with Eventarc lets you trigger an action when a job changes state.”
• Eventarc depends on Pub/Sub From the official documentation: “Eventarc uses Pub/Sub as its transport layer and has a default retry policy that might not work well for your function.”
• BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
• BigQuery works with VPC Service Controls From the official documentation: “BigQuery integrates with VPC Service Controls to restrict access to your BigQuery resources from specific networks.”
• Cloud Audit Logs secures BigQuery From the official documentation: “For BigQuery, [Cloud Audit Logs](/logging/docs/audit) is used for both data access logs and audit logs, which are enabled by default.”
• Cloud Audit Logs works with Cloud Logging From the official documentation: “- For real-time analysis, alerting, and visualization, stream the audit logs to [Logging](/logging/docs/overview).”
• Cloud Audit Logs is part of Eventarc From the official documentation: “Cloud Audit Logs for Eventarc let you track administrative and data-access operations on your resources to help you monitor access and configuration changes.”
• Cloud Audit Logs works with Eventarc From the official documentation: “Create an Eventarc trigger for a Cloud Audit Logs event”
• Cloud Logging is an alternative to Cloud Monitoring From the official documentation: “To see the status of your invalidation request or find a definitive list of recently submitted requests, you can use Cloud Logging or Cloud Monitoring.”
• Cloud Logging works with Dataflow From the official documentation: “You can control the volume of Dataflow logs ingested by Cloud Logging by following the [Filter logs documentation](/dataflow/docs/guides/filter-logs).”
• Cloud Logging works with VPC Service Controls From the official documentation: “This page describes how you can find VPC Service Controls errors using Cloud Logging.”
• Cloud Monitoring works with Eventarc From the official documentation: “Eventarc automatically sends audit logs to Cloud Logging for troubleshooting and streams metrics to Cloud Monitoring for trend analysis and alerting.”
• Cloud Monitoring works with VPC Service Controls From the official documentation: “Cloud Monitoring performs the VPC perimeter check when a project is *added* to a metrics scope:”
• Dataflow works with VPC Service Controls From the official documentation: “To customize DNS resolution when using Dataflow with VPC Service Controls, use Cloud DNS [private zones](/dns/docs/overview#dns-private-zones) instead of using custom BIND servers.”
• Eventarc is an alternative to Pub/Sub From the official documentation: “Instead of using Eventarc, you can use Pub/Sub to directly push messages to your service.”
• VPC Service Controls secures Cloud Monitoring From the official documentation: “VPC Service Controls provides additional security for Cloud Monitoring to help mitigate the risk of data exfiltration.”
• VPC Service Controls secures Dataflow From the official documentation: “Dataflow VPC Service Controls help protect against accidental or targeted action by external entities or by insider entities, which helps to minimize unwarranted data exfiltration risks.”
• VPC Service Controls secures Eventarc From the official documentation: “You can use VPC Service Controls with Eventarc to help protect your services.”
• VPC Service Controls secures Pub/Sub From the official documentation: “If you use [VPC Service Controls](/vpc-service-controls/docs/overview) and your Pub/Sub topic is part of a project inside a service perimeter, you must grant access to projects in order to create notifications.”