A data engineering team is establishing the data collection and preparation architecture for a machine learning project on Google Cloud. The architecture must satisfy three specific requirements: Transform and cleanse large batches of raw data at scale using parallel processing pipelines. Store the prepared structured feature tables in an analytical warehouse where models can also be trained directly using SQL. Archive the original raw input files long-term in a highly durable, cost-effective object store. Which combination of Google Cloud services should the team select?

Dataflow for data processing, BigQuery for data storage and SQL-based ML training, and Cloud Storage for data archival: ### **What the selected services are:** This architecture leverages **Dataflow**, **BigQuery**, and **Cloud Storage**, which represent Google Cloud's core services for building robust data collection, preparation, and analytical pipelines for machine learning workflows. ### **How it meets the requirements:** - **Data processing at scale:** **Dataflow** provides a fully managed, serverless execution environment for **Apache Beam** pipelines. It automates provisioning, scaling, and fault tolerance to cleanse, transform, and normalize massive batch and streaming datasets for downstream machine learning tasks. - **Structured storage and SQL-based ML:** **BigQuery** serves as an enterprise serverless data warehouse designed for structured datasets and fast analytical queries. With **BigQuery ML**, data practitioners can define, train, evaluate, and deploy machine learning models directly using standard **SQL queries** without needing to extract data into external compute environments. - **Cost-effective file archival:** **Cloud Storage** provides highly scalable, highly durable, and cost-effective **object storage**. It is the recommended standard for retaining raw files, datasets, model weights, and training checkpoints long-term. ### **Key advantages:** - **Unified serverless ecosystem:** All three services scale automatically based on data volume, eliminating infrastructure management overhead. - **Native integrations:** Dataflow can ingest directly from Cloud Storage and sink transformed feature tables into BigQuery seamlessly. - **Built-in ML capabilities:** BigQuery ML reduces operational complexity by allowing model training directly where structured data resides. - **Storage lifecycle management:** Cloud Storage allows automated tiering to archival classes (such as Nearline, Coldline, and Archive) to minimize retention costs. ### **Why this is optimal:** This combination establishes the standard Google Cloud reference architecture for ML data preparation. It cleanly separates raw data lake storage (**Cloud Storage**), distributed data transformation execution (**Dataflow**), and high-performance structured feature serving with built-in modeling (**BigQuery**).