A data engineering team is establishing the data collection and preparation architecture for a machine learning project on Google Cloud. The architecture must satisfy three specific requirements: Transform and cleanse large batches of raw data at scale using parallel processing pipelines. Store the prepared structured feature tables in an analytical warehouse where models can also be trained directly using SQL. Archive the original raw input files long-term in a highly durable, cost-effective object store. Which combination of Google Cloud services should the team select?

Dataflow for data processing, BigQuery for data storage and SQL-based ML training, and Cloud Storage for data archival: What the selected services are: This architecture leverages Dataflow, BigQuery, and Cloud Storage, which represent Google Cloud's core services for building robust data collection, preparation, and analytical pipelines for machine learning workflows. How it meets the requirements: Data processing at scale: Dataflow provides a fully managed, serverless execution environment for Apache Beam pipelines. It automates provisioning, scaling, and fault tolerance to cleanse, transform, and normalize massive batch and streaming datasets for downstream machine learning tasks. Structured storage and SQL-based ML: BigQuery serves as an enterprise serverless data warehouse designed for structured datasets and fast analytical queries. With BigQuery ML, data practitioners can define, train, evaluate, and deploy machine learning models directly using standard SQL queries without needing to extract data into external compute environments. Cost-effective file archival: Cloud Storage provides highly scalable, highly durable, and cost-effective object storage. It is the recommended standard for retaining raw files, datasets, model weights, and training checkpoints long-term. Key advantages: Unified serverless ecosystem: All three services scale automatically based on data volume, eliminating infrastructure management overhead. Native integrations: Dataflow can ingest directly from Cloud Storage and sink transformed feature tables into BigQuery seamlessly. Built-in ML capabilities: BigQuery ML reduces operational complexity by allowing model training directly where structured data resides. Storage lifecycle management: Cloud Storage allows automated tiering to archival classes (such as Nearline, Coldline, and Archive) to minimize retention costs. Why this is optimal: This combination establishes the standard Google Cloud reference architecture for ML data preparation. It cleanly separates raw data lake storage (Cloud Storage), distributed data transformation execution (Dataflow), and high-performance structured feature serving with built-in modeling (BigQuery).

Associate Data Practitioner

Unlock the power of your data in the cloud! Get hands-on with Google Cloud's core data services like BigQuery and Looker to validate your practical skills in data ingestion, analysis, and management, and earn your Associate Data Practitioner certification!

Fundamental

Prepare and test your skills

Fundamental

Prepare and test your skills

Dataflow Processing with BigQuery and Cloud Storage

Section: Plan a standard ML project (e.g., data collection, model training, model evaluation, prediction)

...

Explanations

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

FundamentalAssociate Data Practitioner

A multinational financial services company is re-architecting its core payment processing and ledger management system on Google Cloud. The system architecture must satisfy the following technical requirements: Provide globally distributed ACID transactions across multiple geographic regions with external consistency. Colocate customer account records with their associated payment transaction ledger records on storage splits to optimize join queries and eliminate cross-node communication. Prevent hotspotting on storage splits during high-throughput, write-intensive transactional workloads. Which database solution and schema design strategy should the company implement?

Defining architecture to support data access patterns

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

Associate Data Practitioner

Unlock the power of your data in the cloud! Get hands-on with Google Cloud's core data services like BigQuery and Looker to validate your practical skills in data ingestion, analysis, and management, and earn your Associate Data Practitioner certification!

Fundamental

Prepare and test your skills

Fundamental

Prepare and test your skills

Dataflow Processing with BigQuery and Cloud Storage

Section: Plan a standard ML project (e.g., data collection, model training, model evaluation, prediction)

A data engineering team is establishing the data collection and preparation architecture for a machine learning project on Google Cloud. The architecture must satisfy three specific requirements:

Transform and cleanse large batches of raw data at scale using parallel processing pipelines.
Store the prepared structured feature tables in an analytical warehouse where models can also be trained directly using SQL.
Archive the original raw input files long-term in a highly durable, cost-effective object store.

Which combination of Google Cloud services should the team select?

...

Dataplex for data processing, Cloud SQL for data storage and SQL-based ML training, and Transfer Appliance for data archival

...

BigQuery Data Transfer Service for data processing, Dataproc Metastore for data storage and SQL-based ML training, and Managed Lustre for data archival

...

Dataflow for data processing, BigQuery for data storage and SQL-based ML training, and Cloud Storage for data archival

...

Pub/Sub for data processing, Cloud Spanner for data storage and SQL-based ML training, and Filestore for data archival

...

Explanations

What the selected services are:

This architecture leverages Dataflow, BigQuery, and Cloud Storage, which represent Google Cloud's core services for building robust data collection, preparation, and analytical pipelines for machine learning workflows.

How it meets the requirements:

Data processing at scale: Dataflow provides a fully managed, serverless execution environment for Apache Beam pipelines. It automates provisioning, scaling, and fault tolerance to cleanse, transform, and normalize massive batch and streaming datasets for downstream machine learning tasks.
Structured storage and SQL-based ML: BigQuery serves as an enterprise serverless data warehouse designed for structured datasets and fast analytical queries. With BigQuery ML, data practitioners can define, train, evaluate, and deploy machine learning models directly using standard SQL queries without needing to extract data into external compute environments.
Cost-effective file archival: Cloud Storage provides highly scalable, highly durable, and cost-effective object storage. It is the recommended standard for retaining raw files, datasets, model weights, and training checkpoints long-term.

Key advantages:

Unified serverless ecosystem: All three services scale automatically based on data volume, eliminating infrastructure management overhead.
Native integrations: Dataflow can ingest directly from Cloud Storage and sink transformed feature tables into BigQuery seamlessly.
Built-in ML capabilities: BigQuery ML reduces operational complexity by allowing model training directly where structured data resides.
Storage lifecycle management: Cloud Storage allows automated tiering to archival classes (such as Nearline, Coldline, and Archive) to minimize retention costs.

Why this is optimal:

This combination establishes the standard Google Cloud reference architecture for ML data preparation. It cleanly separates raw data lake storage (Cloud Storage), distributed data transformation execution (Dataflow), and high-performance structured feature serving with built-in modeling (BigQuery).

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

FundamentalAssociate Data Practitioner

BigQuery Dataset Prerequisite

Plan a standard ML project (e.g., data collection, model training, model evaluation, prediction)

FundamentalAssociate Data Practitioner

BigQuery ML Model Lifecycle

Plan a standard ML project (e.g., data collection, model training, model evaluation, prediction)

FundamentalAssociate Data Practitioner

Eventarc Trigger Cloud Run

Use Eventarc triggers in event-driven pipelines (Dataform, Dataflow, Cloud Functions, Cloud Run, Cloud Composer)

FundamentalAssociate Data Practitioner

Cloud Storage Dual-Region

Distinguish between primary and secondary data storage location type (e.g., regions, dual-regions, multi-regions, zones) for data redundancy

FundamentalAssociate Data Practitioner

Cloud Storage Regional Location

Choose the appropriate data storage location type (e.g., regional, dual-regional, multi-regional, zonal)

AdvancedProfessional Cloud Data Engineer

A multinational financial services company is re-architecting its core payment processing and ledger management system on Google Cloud. The system architecture must satisfy the following technical requirements: Provide globally distributed ACID transactions across multiple geographic regions with external consistency. Colocate customer account records with their associated payment transaction ledger records on storage splits to optimize join queries and eliminate cross-node communication. Prevent hotspotting on storage splits during high-throughput, write-intensive transactional workloads. Which database solution and schema design strategy should the company implement?

Defining architecture to support data access patterns

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

A data engineering team is establishing the data collection and preparation architecture for a machine learning project on Google Cloud. The architecture must satisfy three specific requirements:

Transform and cleanse large batches of raw data at scale using parallel processing pipelines.
Store the prepared structured feature tables in an analytical warehouse where models can also be trained directly using SQL.
Archive the original raw input files long-term in a highly durable, cost-effective object store.