Prepare and test your skills

Prepare and test your skills

Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An enterprise maintains hundreds of terabytes of clickstream logs stored in Amazon S3 within the us-east-1 region. The data analytics team needs to combine and analyze this remote data alongside core customer demographic tables residing natively in BigQuery in the US multi-region.
The solution must satisfy the following constraints:
Which architecture should the team implement?
The EXTERNAL_QUERY function in BigQuery is designed for federated queries against operational relational databases such as Cloud SQL, AlloyDB, and Cloud Spanner, or SAP Datasphere.
EXTERNAL_QUERY requires a supported relational database engine endpoint and cannot query raw object storage buckets (like Amazon S3 or Cloud Storage) directly.EXTERNAL_QUERY sends SQL statements formatted in the source database's dialect (e.g., PostgreSQL, MySQL), which is not applicable to file-based object stores without a database query layer.The analytical query needed to read live transactional operational tables residing inside a Cloud SQL, AlloyDB, or Cloud Spanner database instance.
This architecture uses Storage Transfer Service to bulk-copy objects from Amazon S3 to Cloud Storage, followed by a Dataflow pipeline that ingests and transforms the staged raw files into native BigQuery managed tables.
The goal were a permanent, one-time or bulk migration of raw data into Google Cloud storage to decommission the AWS infrastructure entirely.
This approach deploys an Apache Spark cluster on Cloud Dataproc in Google Cloud, using the open-source Hadoop S3A connector to stream data directly from AWS S3 over the network and join it with BigQuery tables using the Spark-BigQuery connector.
The workload required custom, non-SQL machine learning or complex graph algorithms implemented in Scala or PySpark that cannot be expressed in standard SQL.
BigQuery Omni is an extension of Google Cloud's serverless analytics engine that runs the BigQuery query engine directly inside other cloud environments, such as Amazon Web Services (AWS) and Microsoft Azure. Combined with BigLake, it allows users to create external tables over structured and semi-structured data stored in Amazon S3 or Azure Blob Storage.
us-east-1). When performing cross-cloud queries, filtering and initial aggregations are executed within the BigQuery Omni AWS compute layer using local slots, transferring only the necessary intermediate or final result rows/columns (via a temporary table mechanism) over the network to the BigQuery region in Google Cloud.bq CLI, APIs, BI Engine) seamlessly.This approach avoids raw bulk data replication across cloud boundaries, dramatically lowering network transit fees and latency while keeping governance and security centralized in Google Cloud.
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
Keep the momentum going with these hand-picked practice scenarios
Last updated Validated