data lineage

Related topics

• BigQuery works with data lineage From the official documentation: “Use data lineage to identify unused assets and optimize costs in your BigQuery project.”
• Cloud Storage works with data lineage From the official documentation: “Data lineage is available for all Managed Service for Apache Spark jobs except SparkR and Spark streaming jobs, and supports BigQuery and Cloud Storage data sources.”
• data lineage is part of Dataflow From the official documentation: “Data lineage is a Dataflow feature that lets you track how data moves through your systems: where it comes from, where it is passed to, and what transformations are applied to it.”
• data lineage is part of Knowledge Catalog From the official documentation: “About data lineage visualization in Knowledge Catalog”
• data lineage works with Managed Service for Apache Airflow From the official documentation: “[Data lineage integration](/composer/docs/composer-2/lineage-integration) is available in Managed Airflow (Gen 2) versions 2.1.2 and later with Airflow versions 2.2.5 and later.”
• data lineage works with Managed Service for Apache Spark From the official documentation: “Learn how to [use data lineage with Managed Service for Apache Spark](/managed-spark/docs/guides/data-lineage).”
• data lineage works with Model Context Protocol From the official documentation: “This page explains how to connect data lineage to developer tools such as the Gemini CLI and other Model Context Protocol (MCP) clients.”
• data lineage works with Vertex AI From the official documentation: “Other integrated systems (such as Cloud Data Fusion and Vertex AI) support table-level lineage only.”
• VPC Service Controls secures data lineage From the official documentation: “Data lineage as part of Knowledge Catalog offers VPC-SC support.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
• BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
• Cloud Logging works with customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• customer-managed encryption keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Dataflow works with Managed Service for Apache Airflow From the official documentation: “Launch Dataflow pipelines with Cloud Composer”
• BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Cloud Logging works with Managed Service for Apache Airflow From the official documentation: “You can use [Cloud Monitoring](/monitoring/docs) and [Cloud Logging](/logging/docs) with Managed Service for Apache Airflow.”
• Cloud Storage works with Managed Service for Apache Airflow From the official documentation: “Managed Airflow associates a Cloud Storage bucket with your environment.”
• customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
• customer-managed encryption keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
• BigQuery works with Managed Service for Apache Airflow From the official documentation: “You can use Managed Airflow to orchestrate services in your data pipelines, such as triggering a job in BigQuery or starting a Dataflow pipeline.”
• Cloud Monitoring works with Managed Service for Apache Airflow From the official documentation: “You can use [Cloud Monitoring](/monitoring/docs) and [Cloud Logging](/logging/docs) with Managed Service for Apache Airflow.”
• Cloud Storage works with customer-managed encryption keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
• Cloud Storage works with Identity and Access Management From the official documentation: “To learn how to use IAM with Cloud Storage, see [Using IAM Permissions](/storage/docs/access-control/using-iam-permissions).”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
• BigQuery is an alternative to Managed Service for Apache Spark From the official documentation: “Run queries against your federated data using BigQuery or Managed Service for Apache Spark.”
• Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
• Cloud Monitoring works with Dataflow From the official documentation: “Dataflow is fully integrated with Cloud Monitoring.”
• Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
• customer-managed encryption keys secures Managed Service for Apache Airflow From the official documentation: “This page describes how to use [Customer Managed Encryption Keys (CMEK)](/kms/docs/cmek) to protect Managed Airflow environments.”
• Dataflow is an alternative to Managed Service for Apache Spark From the official documentation: “For more complex data loads into your Cloud SQL for PostgreSQL database, consider using Dataflow or Managed Service for Apache Spark, which involves creating an ETL process.”
• Identity and Access Management secures BigQuery From the official documentation: “Access to BigQuery data from AlloyDB is managed through IAM.”
• Identity and Access Management secures Cloud Storage From the official documentation: “Cloud Storage uses Identity and Access Management (IAM) for authorization.”
• VPC Service Controls secures Managed Service for Apache Airflow From the official documentation: “Managed Airflow can be selected as a secured service inside a VPC Service Controls perimeter.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• BigQuery works with Identity and Access Management From the official documentation: “BigQuery uses Identity and Access Management (IAM) for authorization.”
• Cloud Monitoring works with Pub/Sub From the official documentation: “Use Cloud Monitoring and Pub/Sub to set up custom monitoring and alerting policies for exported logs.”
• Identity and Access Management secures Vertex AI From the official documentation: “A Vertex AI model with an active endpoint that you have Identity and Access Management (IAM) permission to access.”
• Managed Service for Apache Airflow works with VPC Service Controls From the official documentation: “When used with Managed Airflow, VPC Service Controls have several [known limitations](/vpc-service-controls/docs/supported-products#composer).”
• Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
• BigQuery works with Vertex AI From the official documentation: “The page provides guidance for getting predictions from Vertex AI, based on retail [data exported into BigQuery](/retail/docs/export-data-into-bq).”
• Cloud Logging works with Cloud Monitoring From the official documentation: “If a log entry matches the filter of the alerting policy, then Cloud Logging extracts labels and forwards the data to Cloud Monitoring.”
• Cloud Logging works with Dataflow From the official documentation: “Information stored in [Cloud Logging](/logging/docs/region-support) is primarily generated by the code in your Dataflow program.”
• Cloud Logging is an alternative to Google Cloud Storage From the official documentation: “When you specify your own service account for builds, you must store your build logs in a Cloud Logging bucket, in a user-created Cloud Storage bucket, or in a Google Cloud-created and user-owned Cloud Storage bucket.”
• customer-managed encryption keys secures Cloud Logging From the official documentation: “Cloud Logging supports encrypting Log storage with CMEK keys.”
• customer-managed encryption keys is part of Cloud Storage From the official documentation: “[Cloud Storage](/storage/docs/introduction) provides features such as customer-managed encryption, lifecycle management, and aging and retention policies.”
• customer-managed encryption keys secures Managed Service for Apache Spark From the official documentation: “For data encryption at rest, use the [Cloud Key Management Service](/kms/docs/key-management-service) (KMS) or [Customer Managed Encryption Keys](/managed-spark/docs/concepts/configuring-clusters/customer-managed-encryption) (CMEK).”