Managed Service for Apache Spark

Related topics

• Cloud Data Fusion works with Managed Service for Apache Spark From the official documentation: “Deployed Cloud Data Fusion pipelines run on Managed Service for Apache Spark clusters by default.”
• Managed Service for Apache Spark depends on Private Google Access From the official documentation: “Managed Service for Apache Spark batch workloads and interactive sessions run on VMs with internal IP addresses only and on a regional subnet with [Private Google Access (PGA)](/vpc/docs/configure-private-google-access) enabled automatically on the subnet.”
• BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
• BigQuery is an alternative to Managed Service for Apache Spark From the official documentation: “Run queries against your federated data using BigQuery or Managed Service for Apache Spark.”
• Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
• Dataflow is an alternative to Managed Service for Apache Spark From the official documentation: “For more complex data loads into your Cloud SQL for PostgreSQL database, consider using Dataflow or Managed Service for Apache Spark, which involves creating an ETL process.”
• Application Default Credentials works with Managed Service for Apache Spark From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
• Application Default Credentials secures Managed Service for Apache Spark From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
• Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
• Managed Service for Apache Spark works with Private Google Access From the official documentation: “Managed Service for Apache Spark automatically enables Private Google Access on the regional subnet used by Managed Service for Apache Spark clusters created with image version `2.2` and later.”
• autoscaling is part of Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark [Autoscaling](/managed-spark/docs/concepts/configuring-clusters/autoscaling) dynamically adds and removes Managed Service for Apache Spark workers from a cluster to help ensure that Spark jobs have the resources needed to complete quickly.”
• Cloud Profiler works with Managed Service for Apache Spark From the official documentation: “Profiler supports only Managed Service for Apache Spark Hadoop and Spark job types (Spark, PySpark, SparkSql, and SparkR).”
• Compute Engine works with Managed Service for Apache Spark From the official documentation: “Yes, Managed Service for Apache Spark has native and automatic integrations with Compute Engine, Cloud Storage, Bigtable, BigQuery, Logging, and Cloud Monitoring.”
• customer-managed encryption keys secures Managed Service for Apache Spark From the official documentation: “For data encryption at rest, use the [Cloud Key Management Service](/kms/docs/key-management-service) (KMS) or [Customer Managed Encryption Keys](/managed-spark/docs/concepts/configuring-clusters/customer-managed-encryption) (CMEK).”
• Managed Service for Apache Spark depends on Application Default Credentials From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
• BigQuery connector is part of Managed Service for Apache Spark From the official documentation: “All Managed Service for Apache Spark clusters come with the [BigQuery connector for Hadoop](https://github.com/GoogleCloudDataproc/hadoop-connectors) built in.”
• BigQuery connector works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark integrates with the [BigQuery connector](/dataproc/docs/concepts/connectors/bigquery), a Java library that enables Hadoop and Spark to directly write data to BigQuery by using abstracted versions of the Apache Hadoop…”
• Bigtable works with Managed Service for Apache Spark From the official documentation: “With Managed Service for Apache Spark, you can run Hadoop jobs that read from and write to Bigtable.”
• Cloud Data Fusion depends on Managed Service for Apache Spark From the official documentation: “Cloud Data Fusion uses Managed Service for Apache Spark as the execution environment for pipelines.”
• Cloud Logging works with Managed Service for Apache Spark From the official documentation: “Use [Cloud Logging](/logging) to view logs from the Managed Service for Apache Spark Autoscaler.”
• Cloud Storage connector is part of Managed Service for Apache Spark From the official documentation: “Hadoop and Spark are pre-installed on Managed Service for Apache Spark clusters, and they are configured with the Cloud Storage connector, which allows your code to read and write data directly from and to Cloud Storage.”
• Cloud Storage connector works with Managed Service for Apache Spark From the official documentation: “You can use the Google Cloud CLI to create a Managed Service for Apache Spark cluster and enable the Cloud Storage connector to perform the folder operations.”
• GKE works with Managed Service for Apache Spark From the official documentation: “You can create Managed Service for Apache Spark clusters on: - [Managed Service for Apache Spark clusters on Compute Engine](/dataproc/docs/guides/create-cluster) - [Managed Service for Apache Spark clusters on GKE](/dataproc/docs/guides/dpgke/dataproc-gke-overview)”
• Managed Service for Apache Spark depends on Compute Engine From the official documentation: “Managed Service for Apache Spark clusters are built on [Compute Engine](/compute) instances.”
• Managed Service for Apache Spark configures Private Google Access From the official documentation: “Managed Service for Apache Spark enables [Private Google Access (PGA)](/vpc/docs/private-google-access) on the specified subnet.”
• VPC Service Controls secures Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark requires <a href="/vpc-service-controls/docs/set-up-private-connectivity">special steps</a> to protect using VPC Service Controls.”
• Application Default Credentials works with Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
• Application Default Credentials secures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
• BigQuery depends on Application Default Credentials From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
• Dataflow depends on Application Default Credentials From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
• Application Default Credentials works with BigQuery From the official documentation: “Create a BigQuery client using application default credentials.”
• Application Default Credentials works with Bigtable From the official documentation: “To authenticate to Bigtable, set up Application Default Credentials.”
• Application Default Credentials secures Bigtable From the official documentation: “To authenticate to Bigtable, set up Application Default Credentials.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Application Default Credentials works with Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
• Application Default Credentials works with Compute Engine From the official documentation: “To authenticate to Compute Engine, set up Application Default Credentials.”
• Bigtable depends on Application Default Credentials From the official documentation: “To authenticate to Bigtable, set up Application Default Credentials.”
• VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
• Application Default Credentials secures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
• Application Default Credentials secures Spanner From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
• Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
• Application Default Credentials works with Spanner From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
• Cloud Logging works with customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• Spanner depends on Application Default Credentials From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
• BigQuery works with Bigtable From the official documentation: “External BigQuery data source backed by Bigtable”
• Cloud Run works with Cloud Storage From the official documentation: “Route Cloud Storage events to Cloud Run”
• Application Default Credentials secures Compute Engine From the official documentation: “To authenticate to Compute Engine, set up Application Default Credentials.”
• Cloud Logging works with GKE From the official documentation: “Enable Cloud Logging on GKE Clusters”
• Compute Engine is an alternative to GKE From the official documentation: “To use TPU7x, you can use Google Kubernetes Engine (GKE) or Compute Engine.”
• BigQuery works with BigQuery connector From the official documentation: “To use the output, connect the app to a **BigQuery** connector.”
• BigQuery works with Spanner From the official documentation: “Reverse ETL (BigQuery to Spanner)”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• customer-managed encryption keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Dataflow works with Spanner From the official documentation: “Use Dataflow to load data into Spanner.”
• Cloud Run depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Run, set up Application Default Credentials.”
• Cloud Run is an alternative to GKE From the official documentation: “The app uses Cloud SQL, Vertex AI, and either Google Kubernetes Engine (GKE) or Cloud Run.”
• customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”