An enterprise maintains hundreds of terabytes of clickstream logs stored in Amazon S3 within the us-east-1 region. The data analytics team needs to combine and analyze this remote data alongside core customer demographic tables residing natively in BigQuery in the US multi-region. The solution must satisfy the following constraints: Minimize cross-cloud egress charges and network bandwidth consumption. Eliminate the operational overhead of deploying, managing, or scheduling ETL/ELT pipelines. Enforce unified, fine-grained access governance across cloud providers using standard identity delegation. Which architecture should the team implement?

Create a BigQuery Omni external connection using an AWS IAM role for access delegation, define BigLake external tables referencing the S3 datasets, and execute cross-cloud queries that filter and aggregate data in the AWS region before transferring the result set to Google Cloud.: ### What BigQuery Omni and BigLake are **BigQuery Omni** is an extension of Google Cloud's serverless analytics engine that runs the BigQuery query engine directly inside other cloud environments, such as **Amazon Web Services (AWS)** and **Microsoft Azure**. Combined with **BigLake**, it allows users to create external tables over structured and semi-structured data stored in Amazon S3 or Azure Blob Storage. ### How it meets the requirements - **Minimizes egress costs**: BigQuery Omni executes computation locally where the data resides in AWS (`us-east-1`). When performing cross-cloud queries, filtering and initial aggregations are executed within the BigQuery Omni AWS compute layer using local slots, transferring only the necessary intermediate or final result rows/columns (via a temporary table mechanism) over the network to the BigQuery region in Google Cloud. - **Eliminates ETL/ELT operational overhead**: There is no need to stand up data migration infrastructure, replication tasks, or intermediate pipeline schedulers. Analytics can be executed directly across multi-cloud storage using standard **GoogleSQL** syntax. - **Enforces secure access delegation**: BigQuery Omni leverages native **AWS IAM roles** with trust policies tied to Google Cloud service accounts. This allows administrators to manage access delegation without sharing permanent AWS access keys or managing complex third-party credentials. ### Key advantages and capabilities - **Decoupled compute and storage**: Maintains BigQuery's core architectural separation while executing compute in external clouds. - **Fine-grained access controls**: Enables centralized table-level, column-level, and row-level security through BigLake without moving raw data files. - **Pushdown optimization**: Filters and projections are applied locally before cross-cloud data movement occurs. - **Standard SQL tooling**: Analysts can use familiar BigQuery interfaces (Google Cloud console, `bq` CLI, APIs, BI Engine) seamlessly. ### Why this is optimal This approach avoids raw bulk data replication across cloud boundaries, dramatically lowering network transit fees and latency while keeping governance and security centralized in Google Cloud.

Bigquery Omni Federated Cross Cloud Analytics | professional-cloud-data-engineer

Bigquery Omni Federated Cross Cloud Analytics

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An enterprise maintains hundreds of terabytes of clickstream logs stored in Amazon S3 within the us-east-1 region. The data analytics team needs to combine and analyze this remote data alongside core customer demographic tables residing natively in BigQuery in the US multi-region.

The solution must satisfy the following constraints:

Minimize cross-cloud egress charges and network bandwidth consumption.
Eliminate the operational overhead of deploying, managing, or scheduling ETL/ELT pipelines.
Enforce unified, fine-grained access governance across cloud providers using standard identity delegation.

Which architecture should the team implement?

Configure Storage Transfer Service to execute hourly scheduled batch syncs of raw S3 clickstream files into a Cloud Storage staging bucket, and trigger a Dataflow pipeline to load the data into BigQuery.

Create a BigQuery Omni external connection using an AWS IAM role for access delegation, define BigLake external tables referencing the S3 datasets, and execute cross-cloud queries that filter and aggregate data in the AWS region before transferring the result set to Google Cloud.

Deploy a Google Cloud Dataproc cluster in Google Cloud configured with the Hadoop S3 connector to read the remote S3 data directly into Spark memory and join it with BigQuery.

Execute BigQuery federated queries using the EXTERNAL_QUERY function over a JDBC connection pointing directly to the Amazon S3 bucket to process records on the fly.

Professional Cloud Data Engineer

Bigquery Omni Federated Cross Cloud Analytics

Related questions