Lakehouse runtime catalog

Related topics

• BigQuery works with Lakehouse runtime catalog From the official documentation: “When querying Lakehouse runtime catalog tables from BigQuery, you use a four-part naming structure, often referred to as **P.C.N.T**:”
• Lakehouse runtime catalog works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark supports only PySpark jobs with borderless Lakehouse Metastore.”
• Apache Iceberg works with Lakehouse runtime catalog From the official documentation: “Learn how to use the Lakehouse runtime catalog with Apache Spark and Apache Iceberg.”
• Cloud Storage works with Lakehouse runtime catalog From the official documentation: “Set up and configure Lakehouse catalog endpoints to unify data silos across AWS S3, Cloud Storage, and AlloyDB.”
• Lakehouse runtime catalog is part of Lakehouse From the official documentation: “*The Lakehouse runtime catalog* is a centralized, serverless metadata service that acts as the single source of truth for Google Cloud's Lakehouse.”
• AlloyDB for PostgreSQL works with Lakehouse runtime catalog From the official documentation: “Set up and configure Lakehouse catalog endpoints to unify data silos across AWS S3, Cloud Storage, and AlloyDB.”
• BigQuery depends on Lakehouse runtime catalog From the official documentation: “BigQuery federates queries to these endpoints using the Google Cloud Lakehouse runtime catalog connection.”
• BigQuery Studio works with Lakehouse runtime catalog From the official documentation: “The Lakehouse runtime catalog acts as the unified hub that connects BigQuery Studio to your external open-format data.”
• credential vending secures Lakehouse runtime catalog From the official documentation: “Credential vending is a security mechanism that helps tighten access control when using the Lakehouse runtime catalog.”
• namespace is part of Lakehouse runtime catalog From the official documentation: “Creating a namespace in a catalog (such as an Apache Iceberg REST catalog or Apache Hive catalog) helps you organize your tables within the *Lakehouse runtime catalog*.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
• BigQuery works with Datastream From the official documentation: “Permissions needed for datastream to write to BigQuery.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Cloud Storage works with Datastream From the official documentation: “When you stream to Cloud Storage, Datastream organizes data by object name and timestamp.”
• BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
• Cloud Logging works with Customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• Customer-managed encryption keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
• Customer-managed encryption keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
• Cloud Storage works with Customer-managed encryption keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• AlloyDB for PostgreSQL works with Datastream From the official documentation: “Datastream works with the following types of PostgreSQL databases: - AlloyDB for PostgreSQL”
• BigQuery is an alternative to Managed Service for Apache Spark From the official documentation: “Run queries against your federated data using BigQuery or Managed Service for Apache Spark.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
• Dataflow is an alternative to Managed Service for Apache Spark From the official documentation: “For more complex data loads into your Cloud SQL for PostgreSQL database, consider using Dataflow or Managed Service for Apache Spark, which involves creating an ETL process.”
• AlloyDB for PostgreSQL works with BigQuery From the official documentation: “BigQuery AlloyDB federation lets BigQuery query data residing in AlloyDB in real time without copying or moving the data.”
• AlloyDB for PostgreSQL works with Cloud Storage From the official documentation: “AlloyDB supports importing and exporting compressed `.gz` files from Cloud Storage.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Lakehouse From the official documentation: “BigLake automatically registers the Iceberg table in BigQuery:”
• Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
• AlloyDB for PostgreSQL works with Knowledge Catalog From the official documentation: “Enable the Knowledge Catalog integration for your AlloyDB cluster”
• AlloyDB for PostgreSQL works with Pub/Sub From the official documentation: “Although AlloyDB doesn't provide a built-in method for automating exports, you can build an automation tool using other Google Cloud components such as Cloud Scheduler, Pub/Sub, and Cloud Run functions.”
• Cloud Logging works with Dataflow From the official documentation: “Information stored in [Cloud Logging](/logging/docs/region-support) is primarily generated by the code in your Dataflow program.”
• credential vending secures Cloud Storage From the official documentation: “With credential vending, the catalog securely issues temporary, table-scoped storage tokens to client engines and BigQuery, so external engines don't need direct IAM permissions on your bucket.”
• Customer-managed encryption keys secures Cloud Logging From the official documentation: “Cloud Logging supports encrypting Log storage with CMEK keys.”
• Customer-managed encryption keys is part of Cloud Storage From the official documentation: “[Cloud Storage](/storage/docs/introduction) provides features such as customer-managed encryption, lifecycle management, and aging and retention policies.”
• Customer-managed encryption keys secures Managed Service for Apache Spark From the official documentation: “For data encryption at rest, use the [Cloud Key Management Service](/kms/docs/key-management-service) (KMS) or [Customer Managed Encryption Keys](/managed-spark/docs/concepts/configuring-clusters/customer-managed-encryption) (CMEK).”
• Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
• Dataflow works with Datastream From the official documentation: “In addition, Datastream supports writing the change event stream into Cloud Storage, and offers streamlined integration with Dataflow templates to build custom workflows for loading data into a wide range of destinations, such as Cloud SQL and Spanner.”
• Dataflow works with Lakehouse From the official documentation: “Importing data from external Iceberg catalogs to Lakehouse using Dataflow has the following limitations:”
• Datastream works with Knowledge Catalog From the official documentation: “You can search for and manage your Datastream resources using Knowledge Catalog, which is a platform for storing, managing, and accessing your metadata.”
• Knowledge Catalog works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark integrates with other Google Cloud products, such as BigQuery and Knowledge Catalog.”
• AlloyDB for PostgreSQL works with Cloud Logging From the official documentation: “AlloyDB processes user-generated audit logs asynchronously before securely forwarding them to Cloud Logging.”
• AlloyDB for PostgreSQL works with Customer-managed encryption keys From the official documentation: “This page describes how to create, configure, and apply customer-managed encryption keys (CMEK) for AlloyDB for PostgreSQL.”
• Apache Iceberg works with Cloud Storage From the official documentation: “This architecture uses open table formats like Iceberg to add data warehousing capabilities, such as transactions and schema evolution, to data in Cloud Storage.”
• Apache Iceberg works with Dataflow From the official documentation: “Write from Dataflow to Apache Iceberg, using the dynamic destinations feature to route incoming records to different Iceberg tables.”
• Apache Iceberg works with Datastream From the official documentation: “This section describes the destinations to which you can stream data using Datastream. Supported destinations are: - BigQuery - Cloud Storage - Apache Iceberg tables.”
• Apache Iceberg works with Managed Service for Apache Spark From the official documentation: “You use Apache Iceberg as the table format, Managed Service for Apache Spark for processing, and the Lakehouse runtime catalog Iceberg REST Catalog for unified metadata management.”
• BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
• BigQuery works with credential vending From the official documentation: “BigQuery supports using [credential vending](/lakehouse/docs/credential-vending) for authentication when reading or writing Iceberg tables in the Lakehouse runtime catalog.”