Vertex AI Pipelines

Related topics

• Cloud Logging works with Vertex AI Pipelines From the official documentation: “Cloud Logging for Vertex AI Pipelines is now generally available ([GA](https://cloud.google.com/products/#product-launch-stages)).”
• Google Cloud Pipeline Components works with Vertex AI Pipelines From the official documentation: “You can use Google Cloud Pipeline Components to define and run ML pipelines in Vertex AI Pipelines and other ML pipeline execution backends conformant with Kubeflow Pipelines.”
• Managed Service for Apache Airflow is an alternative to Vertex AI Pipelines From the official documentation: “The Cloud Build pipeline triggers the orchestrator (Managed Airflow or Vertex AI Pipelines).”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
• Cloud Logging works with Customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• Cloud Logging works with GKE From the official documentation: “Enable Cloud Logging on GKE Clusters”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• Customer-managed encryption keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Dataflow works with Managed Service for Apache Airflow From the official documentation: “Launch Dataflow pipelines with Cloud Composer”
• Cloud Logging works with Managed Service for Apache Airflow From the official documentation: “You can use [Cloud Monitoring](/monitoring/docs) and [Cloud Logging](/logging/docs) with Managed Service for Apache Airflow.”
• Cloud Storage works with Managed Service for Apache Airflow From the official documentation: “Managed Airflow associates a Cloud Storage bucket with your environment.”
• Customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
• BigQuery works with Managed Service for Apache Airflow From the official documentation: “You can use Managed Airflow to orchestrate services in your data pipelines, such as triggering a job in BigQuery or starting a Dataflow pipeline.”
• Cloud Storage works with Customer-managed encryption keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
• VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Customer-managed encryption keys secures Managed Service for Apache Airflow From the official documentation: “This page describes how to use [Customer Managed Encryption Keys (CMEK)](/kms/docs/cmek) to protect Managed Airflow environments.”
• VPC Service Controls secures Managed Service for Apache Airflow From the official documentation: “Managed Airflow can be selected as a secured service inside a VPC Service Controls perimeter.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Managed Service for Apache Airflow depends on GKE From the official documentation: “Managed Airflow runs your workloads on GKE.”
• Managed Service for Apache Airflow works with VPC Service Controls From the official documentation: “When used with Managed Airflow, VPC Service Controls have several [known limitations](/vpc-service-controls/docs/supported-products#composer).”
• Cloud Logging works with Dataflow From the official documentation: “Information stored in [Cloud Logging](/logging/docs/region-support) is primarily generated by the code in your Dataflow program.”
• Customer-managed encryption keys secures Cloud Logging From the official documentation: “Cloud Logging supports encrypting Log storage with CMEK keys.”
• Customer-managed encryption keys is part of Cloud Storage From the official documentation: “[Cloud Storage](/storage/docs/introduction) provides features such as customer-managed encryption, lifecycle management, and aging and retention policies.”
• Customer-managed encryption keys secures GKE From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
• BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
• BigQuery is an alternative to Dataflow From the official documentation: “To automate the EDA process, use BigQuery or a managed data processing service like Dataflow.”
• BigQuery works with GKE From the official documentation: “Subsequently, by integrating BigQuery with GKE, you can use BigQuery to efficiently handle massive datasets (like Google Analytics) and the model's natural language understanding capabilities to generate meaningful insights.”
• BigQuery works with VPC Service Controls From the official documentation: “BigQuery integrates with VPC Service Controls to restrict access to your BigQuery resources from specific networks.”
• Cloud Logging is an alternative to Cloud Storage From the official documentation: “When you specify your own service account for builds, you must store your build logs either in Cloud Logging or in a user-created Cloud Storage bucket.”
• Cloud Logging works with VPC Service Controls From the official documentation: “This page describes how you can find VPC Service Controls errors using Cloud Logging.”
• Cloud Storage works with GKE From the official documentation: “Route Cloud Storage events to GKE”
• Customer-managed encryption keys works with Dataflow From the official documentation: “This sample shows how to use encryption keys managed by the customer, with a Dataflow pipeline.”
• Customer-managed encryption keys secures Dataflow From the official documentation: “You can either create CMEKs manually to protect your Dataflow resources or use Cloud KMS Autokey.”
• Dataflow works with GKE From the official documentation: “Route Dataflow events to GKE”
• Dataflow works with VPC Service Controls From the official documentation: “To customize DNS resolution when using Dataflow with VPC Service Controls, use Cloud DNS [private zones](/dns/docs/overview#dns-private-zones) instead of using custom BIND servers.”
• GKE works with Managed Service for Apache Airflow From the official documentation: “Managed Airflow uses Google Kubernetes Engine service to create, manage and delete environment clusters where Airflow components run.”
• Managed Service for Apache Airflow depends on BigQuery From the official documentation: “This happens because Google Kubernetes Engine depends on BigQuery and Managed Service for Apache Airflow inherits this dependency from the Kubernetes Engine API.”
• VPC Service Controls secures Cloud Logging From the official documentation: “The API for Cloud Logging can be protected by VPC Service Controls and the product can be used normally inside service perimeters.”
• VPC Service Controls secures Dataflow From the official documentation: “Place your Dataflow pipelines inside a security perimeter using VPC Service Controls.”
• VPC Service Controls secures GKE From the official documentation: “The Kubernetes Engine API can be protected by VPC Service Controls and the product can be used normally inside service perimeters.”