Cloud Composer

Related topics

• Apache Airflow works with Cloud Composer From the official documentation: “Cloud Composer is a managed Apache Airflow service that helps you create, schedule, monitor and manage workflows.”
• Cloud Build works with Cloud Composer From the official documentation: “Cloud Build synchronizes your development Managed Airflow environment with these new changes.”
• Cloud Composer works with Cloud Logging From the official documentation: “You can use [Cloud Monitoring](/monitoring/docs) and [Cloud Logging](/logging/docs) with Managed Service for Apache Airflow.”
• Cloud Composer depends on Cloud Monitoring From the official documentation: “Managed Airflow uses global endpoints for Cloud Monitoring (`monitoring.googleapis.com`) because some Monitoring APIs don't fully support regional endpoints.”
• Cloud Composer works with Cloud Monitoring From the official documentation: “You can use [Cloud Monitoring](/monitoring/docs) and [Cloud Logging](/logging/docs) with Managed Service for Apache Airflow.”
• Cloud Composer is an alternative to Cloud Run From the official documentation: “If you already have a Managed Airflow environment, you can orchestrate your DPMS metadata exports and imports by running an Airflow DAG on that environment (as shown in Figure 2). This use of an Airflow DAG would be in place of the Cloud Run method mentioned earlier.”
• Cloud Composer works with Cloud Storage From the official documentation: “When you create an environment, Managed Airflow [creates a Cloud Storage bucket](/composer/docs/composer-1/cloud-storage) and associates this bucket with your environment.”
• Cloud Composer depends on Compute Engine From the official documentation: “Composer location to use. Each Composer location constitutes an independent resource namespace constrained to deploying environments into Compute Engine regions inside this location.”
• Cloud Composer works with DAG From the official documentation: “You'll also upload the DAG to your Managed Airflow environment.”
• Cloud Composer depends on Identity-Aware Proxy From the official documentation: “Managed Airflow (Legacy Gen 1) uses [Identity-Aware Proxy](/iap) for this purpose.”
• Cloud Composer works with Managed Service for Apache Spark From the official documentation: “This tutorial shows how to use Managed Service for Apache Airflow to create an [Apache Airflow](http://airflow.apache.org) DAG (Directed Acyclic Graph) that runs an Apache Hadoop wordcount job on a Managed Service for Apache Spark cluster.”
• Cloud Composer works with Private Service Connect From the official documentation: “In Managed Airflow (Gen 2), you can create environments that [use Private Service Connect](/composer/docs/composer-2/configure-private-service-connect) instead of VPC networks.”
• Cloud Composer works with VPC Service Controls From the official documentation: “- **VPC SC**: To deploy Managed Airflow environments inside a security perimeter, see [Configuring VPC SC](/composer/docs/composer-1/configure-vpc-sc). When used with Managed Airflow, VPC Service Controls have several [known…”
• VPC Service Controls secures Cloud Composer From the official documentation: “If you want to protect your environment with VPC Service Controls, then make sure that both host and service projects are located in the same service perimeter For more information, see [Perimeters for environments with Shared VPC and…”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
• Artifact Registry works with Cloud Build From the official documentation: “1. Builds your container and pushes it to Artifact Registry.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Cloud Run depends on Pub/Sub From the official documentation: “Cloud Run function triggered by Pub/Sub to fetch HYPR MFA logs and write to GCS.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
• Cloud Logging works with GKE From the official documentation: “Enable Cloud Logging on GKE Clusters”
• Cloud Monitoring works with GKE From the official documentation: “Enable Cloud Monitoring on GKE Clusters”
• Cloud Run functions works with Pub/Sub From the official documentation: “Use Cloud Run functions to publish messages on Pub/Sub.”
• BigQuery works with Cloud Run functions From the official documentation: “Learn how to write an HTTP Cloud Run function that submits a query to BigQuery.”
• Cloud Build works with Cloud Storage From the official documentation: “Cloud Build then stores Terraform files and the Terraform State File in a Cloud Storage bucket.”
• Compute Engine is an alternative to GKE From the official documentation: “You can also deploy a third-party model to either Vertex AI Model Garden or self-host on GKE or Compute Engine.”
• Artifact Registry works with Cloud Run From the official documentation: “Cloud Run recommends using Artifact Registry.”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Identity-Aware Proxy secures Cloud Run From the official documentation: “IAP provides a central authorization layer for HTTPS applications deployed to Cloud Run.”
• Cloud Build works with Cloud Run From the official documentation: “Cloud Build - create container images and deploy to Cloud Run.”
• Cloud Logging works with Cloud Monitoring From the official documentation: “You can store and view query-response activity logs in Cloud Logging and set up logs-based monitoring by using Cloud Monitoring.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Run functions works with Cloud Storage From the official documentation: “(GCF translates GCS notifications to HTTP requests internally)”
• Cloud Run functions depends on Pub/Sub From the official documentation: “The Pub/Sub message triggers Cloud Run functions.”
• Cloud Storage works with Compute Engine From the official documentation: “For example, you can create a service account that your Compute Engine instances use to access objects stored in Cloud Storage buckets.”
• Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• GKE works with Identity-Aware Proxy From the official documentation: “If you are running GKE version 1.24 or later, you can configure IAP and GKE by using the Kubernetes Gateway API.”
• VPC Service Controls secures Artifact Registry From the official documentation: “If your VPC Service Controls settings are blocking access to Artifact Registry, you must add the Artifact Registry service to the service perimeter of your VPC Service Controls.”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Logging works with Compute Engine From the official documentation: “Compute Engine sends you Cloud Logging events for scheduled, started, or completed maintenance.”
• Cloud Run works with Cloud Storage From the official documentation: “Receive events from Cloud Storage in a Cloud Run service”
• Cloud Run is an alternative to GKE From the official documentation: “For your containerized websites, you can also deploy them to [Cloud Run](/run) in addition to [using GKE](#kubernetes-engine).”
• Compute Engine works with Managed Service for Apache Spark From the official documentation: “Google Compute Engine provides a runtime environment for all features provided by Managed Service for Apache Spark and Gemini Enterprise Agent Platform.”
• GKE depends on Compute Engine From the official documentation: “For example, GKE deploys clusters on Compute Engine VMs.”
• Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
• VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
• Artifact Registry works with Cloud Run functions From the official documentation: “Artifact Registry lets you store private npm and Python packages that you can include as dependencies in a Cloud Run function.”
• Artifact Registry works with GKE From the official documentation: “In the next tutorial, you deploy those images from the Artifact Registry repository to a GKE cluster:”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
• BigQuery works with Private Service Connect From the official documentation: “Click the private zone that contains the DNS records for the Private Service Connect endpoints, which you created to allow BigQuery Toolkit for SAP to privately connect to the BigQuery and IAM APIs.”
• Cloud Build works with GKE From the official documentation: “Remove the permission granted to Cloud Build to connect to GKE.”
• Cloud Logging works with Cloud Run functions From the official documentation: “Cloud Run functions is configured to use Cloud Logging automatically.”