Dataproc

Related topics

• CMEK secures Dataproc From the official documentation: “Require CMEK on Dataproc Clusters”
• Dataproc works with Google Cloud Storage From the official documentation: “# Dataproc job output is saved to the Cloud Storage bucket”
• BigQuery works with Dataproc From the official documentation: “the built-in BigQuery Connector for Spark that bridges BigQuery and Cloud Dataproc.”
• Cloud Data Fusion works with Dataproc From the official documentation: “Cloud Data Fusion translates the pipeline that you build in the Studio into an Apache Spark program that executes transformations in parallel on an ephemeral Managed Service for Apache Spark cluster.”
• Cloud Storage works with Dataproc From the official documentation: “Dataproc job output gets saved to the Google Cloud Storage bucket # allocated to the job.”
• Customer-Managed Encryption Keys works with Dataproc From the official documentation: “A Dataproc cluster with customer-managed encryption keys (CMEK) gives you more control over data encryption and key management.”
• Dataproc depends on Compute Engine From the official documentation: “Dataproc location to use. Each Dataproc location constitutes an independent resource namespace constrained to deploying instances into Compute Engine zones inside the location.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
• BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Cloud SQL works with Cloud Storage From the official documentation: “Exports data from a Cloud SQL instance to a Cloud Storage bucket as a SQL dump or CSV file.”
• Cloud Run depends on Pub/Sub From the official documentation: “Cloud Run function triggered by Pub/Sub to fetch HYPR MFA logs and write to GCS.”
• Customer-Managed Encryption Keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• Cloud Data Fusion works with Managed Service for Apache Spark From the official documentation: “Deployed Cloud Data Fusion pipelines run on Managed Service for Apache Spark clusters by default.”
• CMEK secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
• Customer-Managed Encryption Keys secures Cloud SQL From the official documentation: “Enable CMEK for Cloud SQL Databases”
• BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
• Cloud Run works with Cloud SQL From the official documentation: “Connecting from Cloud Run (fully managed) to Cloud SQL”
• Cloud SQL works with Compute Engine From the official documentation: “Cloud SQL uses Compute Engine virtual machines (VMs) with persistent storage disks.”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Storage works with Customer-Managed Encryption Keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• CMEK secures BigQuery From the official documentation: “Enable CMEK for BigQuery Tables”
• Cloud Logging works with Cloud SQL From the official documentation: “Database Migration Service and Cloud SQL use Cloud Logging.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Run works with Google Cloud Storage From the official documentation: “The Cloud Run function needs a service account with permissions to write to GCS bucket.”
• Cloud Storage works with Compute Engine From the official documentation: “For example, you can create a service account that your Compute Engine instances use to access objects stored in Cloud Storage buckets.”
• Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• CMEK secures Cloud SQL From the official documentation: “Enable CMEK for Cloud SQL Databases”
• CMEK secures Cloud Storage From the official documentation: “Enable CMEK for Cloud Storage Buckets”
• Customer-Managed Encryption Keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
• Customer-Managed Encryption Keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
• BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Logging works with Compute Engine From the official documentation: “Compute Engine sends you Cloud Logging events for scheduled, started, or completed maintenance.”
• Cloud Run works with Cloud Storage From the official documentation: “Receive events from Cloud Storage in a Cloud Run service”
• Compute Engine works with Managed Service for Apache Spark From the official documentation: “Google Compute Engine provides a runtime environment for all features provided by Managed Service for Apache Spark and Gemini Enterprise Agent Platform.”
• Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
• BigQuery works with Cloud Data Fusion From the official documentation: “Cloud Data Fusion pulls data from the storage bucket and writes it to BigQuery.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
• Cloud Data Fusion works with Cloud Logging From the official documentation: “To use Cloud Logging with your Cloud Data Fusion pipeline, create a Cloud Data Fusion instance with Cloud Logging enabled:”
• Cloud Logging is an alternative to Cloud Storage From the official documentation: “Instead, they are routed to either Cloud Storage (recommended) or Cloud Logging, depending on your organization's configuration.”
• Cloud Logging works with Customer-Managed Encryption Keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• Cloud Logging works with Google Cloud Storage From the official documentation: “Cloud Logging can [route logs](/logging/docs/export/configure_export_v2) to Cloud Storage by Google SecOps on a scheduled basis.”
• Cloud Logging is an alternative to Google Cloud Storage From the official documentation: “When you specify your own service account for builds, you must store your build logs in a Cloud Logging bucket, in a user-created Cloud Storage bucket, or in a Google Cloud-created and user-owned Cloud Storage bucket.”
• Cloud Run works with Pub/Sub From the official documentation: “A Cloud Run app receives the Pub/Sub message.”
• CMEK works with Dataflow From the official documentation: “The region for your CMEK and the [region](/dataflow/docs/resources/locations) for your Dataflow job must be the same.”
• Compute Engine works with Google Cloud Storage From the official documentation: “The usage export bucket is a Google Cloud Storage bucket where Compute Engine usage reports are stored.”
• Customer-Managed Encryption Keys depends on Cloud Storage From the official documentation: “If used for storing Customer-managed Encryption Keys (CMEK): Cloud Storage (See [Cloud Storage pricing](/storage/pricing).)”
• Customer-Managed Encryption Keys secures Compute Engine From the official documentation: “In IL5, CMEK is enabled by default for Compute Engine, Google Kubernetes Engine (GKE), Cloud Storage, and Cloud SQL.”
• Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
• Dataflow depends on Compute Engine From the official documentation: “When Dataflow runs a pipeline, the processing is distributed across multiple Compute Engine virtual machines (VMs), also known as workers.”
• Dataflow works with Google Cloud Storage From the official documentation: “When you run a pipeline using Dataflow, your results are stored in a Cloud Storage bucket.”