An enterprise is planning the migration and modernization of its data processing pipelines to Google Cloud. The solution must address two distinct workload requirements: Workload 1: A large suite of existing legacy Apache Spark and Apache Hadoop MapReduce jobs that need to be migrated quickly with minimal code refactoring and customized open-source software (OSS) dependencies. Workload 2: A new analytics application requiring a unified programming framework to handle both continuous real-time streaming data and nightly batch aggregations with fully automated, serverless infrastructure management. Which combination of data processing services should the cloud architect recommend?

Migrate Workload 1 to Cloud Dataproc using ephemeral clusters; build Workload 2 on Cloud Dataflow using the Apache Beam SDK.: What Cloud Dataproc and Cloud Dataflow are Cloud Dataproc is a managed service for deploying and running open-source data processing engines like Apache Spark, Apache Hadoop, and Apache Flink. Cloud Dataflow is a fully managed, serverless data processing service that executes batch and streaming pipelines written with the Apache Beam SDK. How it meets the requirements Workload 1 (Legacy Hadoop/Spark Migration): Dataproc provides native compatibility with existing Spark and Hadoop ecosystems, allowing fast migration with minimal or no code changes. By leveraging ephemeral clusters, the enterprise can spin up clusters tailored to specific jobs with custom initialization actions and terminate them upon completion, reducing operational overhead and idle infrastructure costs. Workload 2 (Unified Batch & Streaming Serverless): Dataflow and Apache Beam provide a unified programming model where developers write pipeline code once to handle both bounded (batch) and unbounded (streaming) datasets. As a serverless platform, Dataflow manages worker provisioning, autoscaling, step fusion, and dynamic work rebalancing automatically without requiring cluster administration. Key advantages and capabilities Ecosystem Portability: Dataproc maintains 100% open-source software (OSS) API parity for existing tools and libraries. Ephemeral Resource Model: Dataproc allows clusters to be created and destroyed per job, avoiding monolithic cluster maintenance. Unified Programming Model: Apache Beam allows consistent transformation logic across real-time events and historic batches. Zero Infrastructure Management: Dataflow completely abstracts compute infrastructure and optimizes worker distribution automatically. Why this is optimal This architecture pairs the ideal execution environment to each workload's lifecycle: Dataproc eliminates refactoring costs for existing Spark/Hadoop jobs, while Dataflow delivers a modern, serverless, unified batch and stream runtime for the new analytics system.

Professional Cloud Architect

Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam!

Intermediate

Prepare and test your skills

Intermediate

Prepare and test your skills

Dataproc for Spark Migration and Dataflow for Streaming

Section: Choosing data processing solutions

...

Explanations

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

IntermediateProfessional Cloud Architect

A financial enterprise stores regulated transaction records in Cloud Storage. Compliance mandates require that: All uploaded records must adhere to Write-Once-Read-Many (WORM) compliance and cannot be deleted or overwritten by any user or administrator for seven years. Individual transaction records subject to pending legal audits must be indefinitely protected from deletion, even if their standard retention duration expires during an investigation. Audit protections on specific records must be easily removable once legal proceedings conclude, without altering the underlying bucket configuration. Which storage configuration strategy should the cloud architect implement to meet these requirements?

Data retention and data lifecycle management

IntermediateProfessional Cloud Architect

SLIs, SLOs, and Error Budgets

Identifying functional and non-functional requirements

AdvancedProfessional Cloud DevOps Engineer

An organization enforces strict data residency requirements by setting the constraints/gcp.resourceLocations constraint at the organization root with allowedValues set to European regions (in:eu-locations). A specialized data science team requires an exception to deploy new resources in us-central1 within their dedicated Google Cloud project, while maintaining European region access and preventing inheritance conflicts. How should the DevOps engineer configure the organization policy on the target project to grant this exception?

Data residency

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

Professional Cloud Architect

Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam!

Intermediate

Prepare and test your skills

Intermediate

Prepare and test your skills

Dataproc for Spark Migration and Dataflow for Streaming

Section: Choosing data processing solutions

An enterprise is planning the migration and modernization of its data processing pipelines to Google Cloud. The solution must address two distinct workload requirements:

Workload 1: A large suite of existing legacy Apache Spark and Apache Hadoop MapReduce jobs that need to be migrated quickly with minimal code refactoring and customized open-source software (OSS) dependencies.
Workload 2: A new analytics application requiring a unified programming framework to handle both continuous real-time streaming data and nightly batch aggregations with fully automated, serverless infrastructure management.

Which combination of data processing services should the cloud architect recommend?

...

Migrate Workload 1 to Cloud Dataflow using Apache Beam connectors; build Workload 2 on Cloud Dataproc persistent clusters with Spark Streaming.

...

Migrate Workload 1 to BigQuery scheduled queries; build Workload 2 using Datastream with BigQuery change data capture (CDC).

...

Deploy both Workload 1 and Workload 2 onto a single large persistent Compute Engine cluster configured with Apache Flink and Apache Hadoop.

...

Migrate Workload 1 to Cloud Dataproc using ephemeral clusters; build Workload 2 on Cloud Dataflow using the Apache Beam SDK.

...

Explanations

What Cloud Dataproc and Cloud Dataflow are

Cloud Dataproc is a managed service for deploying and running open-source data processing engines like Apache Spark, Apache Hadoop, and Apache Flink. Cloud Dataflow is a fully managed, serverless data processing service that executes batch and streaming pipelines written with the Apache Beam SDK.

How it meets the requirements

Workload 1 (Legacy Hadoop/Spark Migration): Dataproc provides native compatibility with existing Spark and Hadoop ecosystems, allowing fast migration with minimal or no code changes. By leveraging ephemeral clusters, the enterprise can spin up clusters tailored to specific jobs with custom initialization actions and terminate them upon completion, reducing operational overhead and idle infrastructure costs.
Workload 2 (Unified Batch & Streaming Serverless): Dataflow and Apache Beam provide a unified programming model where developers write pipeline code once to handle both bounded (batch) and unbounded (streaming) datasets. As a serverless platform, Dataflow manages worker provisioning, autoscaling, step fusion, and dynamic work rebalancing automatically without requiring cluster administration.

Key advantages and capabilities

Ecosystem Portability: Dataproc maintains 100% open-source software (OSS) API parity for existing tools and libraries.
Ephemeral Resource Model: Dataproc allows clusters to be created and destroyed per job, avoiding monolithic cluster maintenance.
Unified Programming Model: Apache Beam allows consistent transformation logic across real-time events and historic batches.
Zero Infrastructure Management: Dataflow completely abstracts compute infrastructure and optimizes worker distribution automatically.

Why this is optimal

This architecture pairs the ideal execution environment to each workload's lifecycle: Dataproc eliminates refactoring costs for existing Spark/Hadoop jobs, while Dataflow delivers a modern, serverless, unified batch and stream runtime for the new analytics system.

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

IntermediateProfessional Cloud Architect

Cloud Pub/Sub to Dataflow to BigQuery Streaming

Choosing data processing solutions

IntermediateProfessional Cloud Architect

BigQuery ELT and Cloud Dataprep

Choosing data processing solutions

IntermediateProfessional Cloud Architect

Flex-start (Dynamic Workload Scheduler)

Using AI Hypercomputer (e.g., using AI Hypercomputer, Cloud Run functions, and Vertex AI for ML/AI workloads; integrating GPUs and TPUs in ML model training and serving; optimizing for different consumption models; and running large-scale AI model training

IntermediateProfessional Cloud Architect

A financial enterprise stores regulated transaction records in Cloud Storage. Compliance mandates require that: All uploaded records must adhere to Write-Once-Read-Many (WORM) compliance and cannot be deleted or overwritten by any user or administrator for seven years. Individual transaction records subject to pending legal audits must be indefinitely protected from deletion, even if their standard retention duration expires during an investigation. Audit protections on specific records must be easily removable once legal proceedings conclude, without altering the underlying bucket configuration. Which storage configuration strategy should the cloud architect implement to meet these requirements?

Data retention and data lifecycle management

IntermediateProfessional Cloud Architect

SLIs, SLOs, and Error Budgets

Identifying functional and non-functional requirements

AdvancedProfessional Cloud DevOps Engineer

An organization enforces strict data residency requirements by setting the constraints/gcp.resourceLocations constraint at the organization root with allowedValues set to European regions (in:eu-locations). A specialized data science team requires an exception to deploy new resources in us-central1 within their dedicated Google Cloud project, while maintaining European region access and preventing inheritance conflicts. How should the DevOps engineer configure the organization policy on the target project to grant this exception?

Data residency

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

An enterprise is planning the migration and modernization of its data processing pipelines to Google Cloud. The solution must address two distinct workload requirements:

Workload 1: A large suite of existing legacy Apache Spark and Apache Hadoop MapReduce jobs that need to be migrated quickly with minimal code refactoring and customized open-source software (OSS) dependencies.
Workload 2: A new analytics application requiring a unified programming framework to handle both continuous real-time streaming data and nightly batch aggregations with fully automated, serverless infrastructure management.