Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Google Cloud service
Lakehouse
Topic in 2 exam domains
Also called: BigLake, Google Cloud's Lakehouse
A Google Cloud service (formerly BigLake) that provides unified data lakehouse management and interoperability.
Where this appears on the exam
2 objectives in 2 of 5 domains
Domain 3
Storing the data~20% of exam
~15% of exam
Related topics
FoundationsBuilding blocksThis topicWorks alongsideAppliedWorkload Identity FederationFeatureVPC Service ControlsServiceIdentity and Access ManagementServiceApplication Default CredentialsFeatureBigQueryServiceDataflowServiceKnowledge CatalogServiceCloud StorageServicePub/SubServiceSpannerService
LakehouseService
Cloud RunServiceCloud Audit LogsServiceDatastreamServiceSAP Business Data CloudServiceAlloyDB for PostgreSQLServiceLakehouse runtime catalogFeaturePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- BigQuery works with Lakehouse From the official documentation: “BigLake automatically registers the Iceberg table in BigQuery:”
- Dataflow works with Lakehouse From the official documentation: “Importing data from external Iceberg catalogs to Lakehouse using Dataflow has the following limitations:”
- Lakehouse runtime catalog is part of Lakehouse From the official documentation: “*The Lakehouse runtime catalog* is a centralized, serverless metadata service that acts as the single source of truth for Google Cloud's Lakehouse.”
- BigQuery is an alternative to Lakehouse From the official documentation: “The BigLake External Tables view outlines tables that are identified as targets to migration to BigLake instead of BigQuery.”
- Cloud Audit Logs works with Lakehouse From the official documentation: “When you call a method, Lakehouse generates an audit log whose category is dependent on the `type` property of the permission required to perform the method.”
- Cloud Storage works with Lakehouse From the official documentation: “If your Parquet files are in Cloud Storage, this transform registers the files with Lakehouse without moving or rewriting the underlying data.”
- Datastream works with Lakehouse From the official documentation: “- [Learn how to use Datastream to replicate transactional databases to Lakehouse tables](/datastream/docs/destination-blmt).”
- Identity and Access Management secures Lakehouse From the official documentation: “Lakehouse uses Identity and Access Management (IAM) for authorization.”
- Knowledge Catalog secures Lakehouse From the official documentation: “Knowledge Catalog provides centralized security, lineage, and governance policies across your entire lakehouse.”
- Lakehouse works with SAP Business Data Cloud From the official documentation: “The integration between Lakehouse and SAP BDC supports two workflows:”
- Workload Identity Federation secures Lakehouse From the official documentation: “To let SAP BDC access your borderless Lakehouse resources without requiring exported service account keys, configure Workload Identity Federation (WIF).”
- Application Default Credentials works with Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials secures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- BigQuery depends on Application Default Credentials From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- Dataflow depends on Application Default Credentials From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials works with Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
- Application Default Credentials works with BigQuery From the official documentation: “Create a BigQuery client using application default credentials.”
- Application Default Credentials secures Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery works with Datastream From the official documentation: “Permissions needed for datastream to write to BigQuery.”
- Pub/Sub depends on Application Default Credentials From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- Cloud Storage works with Datastream From the official documentation: “When you stream to Cloud Storage, Datastream organizes data by object name and timestamp.”
- Application Default Credentials works with Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
- Application Default Credentials secures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials secures Spanner From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
- Knowledge Catalog depends on Application Default Credentials From the official documentation: “To authenticate to Knowledge Catalog, set up Application Default Credentials.”
- Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials works with Spanner From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
- BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
- Spanner depends on Application Default Credentials From the official documentation: “To authenticate to Spanner, set up Application Default Credentials.”
- Cloud Run works with Pub/Sub From the official documentation: “Route Cloud Pub/Sub events to Cloud Run”
- BigQuery works with Lakehouse runtime catalog From the official documentation: “When querying Lakehouse runtime catalog tables from BigQuery, you use a four-part naming structure, often referred to as **P.C.N.T**:”
- Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
- Cloud Run works with Cloud Storage From the official documentation: “Route Cloud Storage events to Cloud Run”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery works with Spanner From the official documentation: “Reverse ETL (BigQuery to Spanner)”
- Cloud Run depends on Pub/Sub From the official documentation: “Create a Pub/Sub topic that Cloud Scheduler will publish to and the Cloud Run function will subscribe to.”
- Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
- Dataflow works with Spanner From the official documentation: “Use Dataflow to load data into Spanner.”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Cloud Run depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Run, set up Application Default Credentials.”
- Application Default Credentials secures Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials secures Knowledge Catalog From the official documentation: “To authenticate to Knowledge Catalog, set up Application Default Credentials.”
- BigQuery works with Cloud Audit Logs From the official documentation: “Cloud Audit Logs maintain a record of Google Cloud events, including BigQuery events.”
- Cloud Storage works with Identity and Access Management From the official documentation: “To learn how to use IAM with Cloud Storage, see [Using IAM Permissions](/storage/docs/access-control/using-iam-permissions).”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
- AlloyDB for PostgreSQL works with Datastream From the official documentation: “Datastream works with the following types of PostgreSQL databases: - AlloyDB for PostgreSQL”
- Application Default Credentials works with Cloud Run From the official documentation: “To authenticate to Cloud Run, set up Application Default Credentials.”
- Application Default Credentials works with Identity and Access Management From the official documentation: “To authenticate to IAM, set up Application Default Credentials.”
- Application Default Credentials works with Knowledge Catalog From the official documentation: “To authenticate to Knowledge Catalog, set up Application Default Credentials.”
- Cloud Audit Logs works with VPC Service Controls From the official documentation: “VPC Service Controls integrate with Cloud Audit Logs to monitor access to the control plane.”
- Identity and Access Management secures BigQuery From the official documentation: “Access to BigQuery data from AlloyDB is managed through IAM.”
- Identity and Access Management secures Cloud Storage From the official documentation: “Cloud Storage uses Identity and Access Management (IAM) for authorization.”
- AlloyDB for PostgreSQL works with BigQuery From the official documentation: “BigQuery AlloyDB federation lets BigQuery query data residing in AlloyDB in real time without copying or moving the data.”
- AlloyDB for PostgreSQL works with Cloud Storage From the official documentation: “AlloyDB supports importing and exporting compressed `.gz` files from Cloud Storage.”
How learners score on this topic
Last updated