Your enterprise data team is modernizing its analytics infrastructure on Google Cloud and needs to select appropriate managed compute services for two distinct workloads: Workload 1: Migrate existing Apache Spark and Apache Hadoop MapReduce jobs to the cloud with minimal code refactoring while maintaining support for the open-source Hadoop ecosystem. Workload 2: Build a new pipeline using Apache Beam to execute unified batch and real-time streaming transformations with serverless scaling and zero infrastructure management. Which combination of Google Cloud services should you recommend?

Dataproc for Workload 1 and Dataflow for Workload 2: What Dataproc and Dataflow Are Dataproc is Google Cloud's managed service for running open-source data processing engines such as Apache Spark, Apache Hadoop, Apache Flink, and Hive. Dataflow is a fully managed, serverless execution service designed to run Apache Beam pipelines for unified stream (real-time) and batch data processing. How This Solution Meets the Requirements Meets Workload 1 (Spark/Hadoop Migration): Dataproc provides native, fully managed Hadoop and Spark clusters, allowing existing jobs, configurations, and ecosystem tools to run seamlessly in the cloud without requiring major code refactoring. Meets Workload 2 (Unified Beam Pipelines): Dataflow provides fully managed, serverless execution specifically optimized for Apache Beam pipelines. It automatically handles resource provisioning, dynamic autoscaling, and pipeline optimization for both continuous streaming and bounded batch data without requiring cluster administration. Key Advantages and Capabilities Ecosystem Compatibility: Dataproc allows easy lift-and-shift of legacy big data workloads, retaining tooling such as Spark SQL, PySpark, and MapReduce. Serverless Automation: Dataflow automatically provisions worker nodes, scales dynamically based on pipeline throughput, and manages sharding/fusion optimizations. Unified Programming Model: Dataflow and Apache Beam allow developers to write identical processing logic across batch and streaming ingestion paths. Cost Efficiency: Dataproc offers per-second billing with fast cluster creation and ephemeral cluster models, while Dataflow ensures you only pay for compute resources actively consumed during pipeline execution. Why This Is Optimal Choosing Dataproc preserves prior engineering investments in Hadoop/Spark applications, while leveraging Dataflow for new Apache Beam pipelines guarantees true serverless scalability and simplified operations for unified streaming and batch analytics.

Professional Cloud Architect

Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam!

Intermediate

Prepare and test your skills

Intermediate

Prepare and test your skills

Choosing Dataproc for Spark and Dataflow for Beam

Section: Data processing and compute provisioning

...

Explanations

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

IntermediateProfessional Cloud Architect

Your team is building a secure CI/CD pipeline using Cloud Build that packages containerized applications and deploys them to Google Kubernetes Engine (GKE). The pipeline has two distinct secret management requirements: Build-time: The build process must authenticate to a private package repository using an API token to fetch build dependencies without baking the token into the resulting container image. Runtime: The deployed application in GKE must securely access database credentials at startup while enforcing least privilege and avoiding static credential files. Which configuration strategy should you implement to satisfy both requirements?

Build versus runtime secret injection

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

Professional Cloud Architect

Intrigued by the art of cloud architecture? Discover how to design, develop, and manage robust, secure, scalable, and dynamic solutions on Google Cloud as you prepare for the Professional Cloud Architect exam!

Intermediate

Prepare and test your skills

Intermediate

Prepare and test your skills

Choosing Dataproc for Spark and Dataflow for Beam

Section: Data processing and compute provisioning

Your enterprise data team is modernizing its analytics infrastructure on Google Cloud and needs to select appropriate managed compute services for two distinct workloads:

Workload 1: Migrate existing Apache Spark and Apache Hadoop MapReduce jobs to the cloud with minimal code refactoring while maintaining support for the open-source Hadoop ecosystem.
Workload 2: Build a new pipeline using Apache Beam to execute unified batch and real-time streaming transformations with serverless scaling and zero infrastructure management.

Which combination of Google Cloud services should you recommend?

...

Dataflow for Workload 1 and Dataproc for Workload 2

...

Dataproc for Workload 1 and Cloud Data Fusion for Workload 2

...

Dataproc for Workload 1 and Dataflow for Workload 2

...

BigQuery for Workload 1 and Dataproc Metastore for Workload 2

...

Explanations

What Dataproc and Dataflow Are

Dataproc is Google Cloud's managed service for running open-source data processing engines such as Apache Spark, Apache Hadoop, Apache Flink, and Hive. Dataflow is a fully managed, serverless execution service designed to run Apache Beam pipelines for unified stream (real-time) and batch data processing.

How This Solution Meets the Requirements

Meets Workload 1 (Spark/Hadoop Migration): Dataproc provides native, fully managed Hadoop and Spark clusters, allowing existing jobs, configurations, and ecosystem tools to run seamlessly in the cloud without requiring major code refactoring.
Meets Workload 2 (Unified Beam Pipelines): Dataflow provides fully managed, serverless execution specifically optimized for Apache Beam pipelines. It automatically handles resource provisioning, dynamic autoscaling, and pipeline optimization for both continuous streaming and bounded batch data without requiring cluster administration.

Key Advantages and Capabilities

Ecosystem Compatibility: Dataproc allows easy lift-and-shift of legacy big data workloads, retaining tooling such as Spark SQL, PySpark, and MapReduce.
Serverless Automation: Dataflow automatically provisions worker nodes, scales dynamically based on pipeline throughput, and manages sharding/fusion optimizations.
Unified Programming Model: Dataflow and Apache Beam allow developers to write identical processing logic across batch and streaming ingestion paths.
Cost Efficiency: Dataproc offers per-second billing with fast cluster creation and ephemeral cluster models, while Dataflow ensures you only pay for compute resources actively consumed during pipeline execution.

Why This Is Optimal

Choosing Dataproc preserves prior engineering investments in Hadoop/Spark applications, while leveraging Dataflow for new Apache Beam pipelines guarantees true serverless scalability and simplified operations for unified streaming and batch analytics.

...

View source documentation

Keep the momentum going with these hand-picked practice scenarios

IntermediateProfessional Cloud Architect

Cloud Run Scale-to-Zero

Data processing and compute provisioning

IntermediateProfessional Cloud Architect

Spot VMs Custom Machine Types Autoscaling

Data processing and compute provisioning

IntermediateProfessional Cloud Architect

BigQuery Omni With Workload Identity Federation

Integration with on-premises/multicloud environments

IntermediateProfessional Cloud Architect

Vertex AI Model Garden Deployment with Cloud Deploy

Integrating AI models from Model Garden into the solution

IntermediateProfessional Cloud Architect

Uniform Bucket-Level Access

Security and access management

AdvancedProfessional Cloud DevOps Engineer

Your team is building a secure CI/CD pipeline using Cloud Build that packages containerized applications and deploys them to Google Kubernetes Engine (GKE). The pipeline has two distinct secret management requirements: Build-time: The build process must authenticate to a private package repository using an API token to fetch build dependencies without baking the token into the resulting container image. Runtime: The deployed application in GKE must securely access database credentials at startup while enforcing least privilege and avoiding static credential files. Which configuration strategy should you implement to satisfy both requirements?

Build versus runtime secret injection

Want more questions like this?

Get a free certification question every week.

Last updated August 28, 2026

Your enterprise data team is modernizing its analytics infrastructure on Google Cloud and needs to select appropriate managed compute services for two distinct workloads:

Workload 1: Migrate existing Apache Spark and Apache Hadoop MapReduce jobs to the cloud with minimal code refactoring while maintaining support for the open-source Hadoop ecosystem.
Workload 2: Build a new pipeline using Apache Beam to execute unified batch and real-time streaming transformations with serverless scaling and zero infrastructure management.

Which combination of Google Cloud services should you recommend?

Dataflow for Workload 1 and Dataproc for Workload 2

Dataproc for Workload 1 and Cloud Data Fusion for Workload 2

Dataproc for Workload 1 and Dataflow for Workload 2

BigQuery for Workload 1 and Dataproc Metastore for Workload 2

Dataflow for Workload 1 and Dataproc for Workload 2

Dataproc for Workload 1 and Cloud Data Fusion for Workload 2

Dataproc for Workload 1 and Dataflow for Workload 2

What Dataproc and Dataflow Are

Dataproc is Google Cloud's managed service for running open-source data processing engines such as Apache Spark, Apache Hadoop, Apache Flink, and Hive. Dataflow is a fully managed, serverless execution service designed to run Apache Beam pipelines for unified stream (real-time) and batch data processing.

How This Solution Meets the Requirements

Meets Workload 1 (Spark/Hadoop Migration): Dataproc provides native, fully managed Hadoop and Spark clusters, allowing existing jobs, configurations, and ecosystem tools to run seamlessly in the cloud without requiring major code refactoring.
Meets Workload 2 (Unified Beam Pipelines): Dataflow provides fully managed, serverless execution specifically optimized for Apache Beam pipelines. It automatically handles resource provisioning, dynamic autoscaling, and pipeline optimization for both continuous streaming and bounded batch data without requiring cluster administration.

Key Advantages and Capabilities

Ecosystem Compatibility: Dataproc allows easy lift-and-shift of legacy big data workloads, retaining tooling such as Spark SQL, PySpark, and MapReduce.
Serverless Automation: Dataflow automatically provisions worker nodes, scales dynamically based on pipeline throughput, and manages sharding/fusion optimizations.
Unified Programming Model: Dataflow and Apache Beam allow developers to write identical processing logic across batch and streaming ingestion paths.
Cost Efficiency: Dataproc offers per-second billing with fast cluster creation and ephemeral cluster models, while Dataflow ensures you only pay for compute resources actively consumed during pipeline execution.

Why This Is Optimal

Choosing Dataproc preserves prior engineering investments in Hadoop/Spark applications, while leveraging Dataflow for new Apache Beam pipelines guarantees true serverless scalability and simplified operations for unified streaming and batch analytics.

BigQuery for Workload 1 and Dataproc Metastore for Workload 2

Professional Cloud Architect

Choosing Dataproc for Spark and Dataflow for Beam

Related questions

Cloud Run Scale-to-Zero

Spot VMs Custom Machine Types Autoscaling

BigQuery Omni With Workload Identity Federation

Vertex AI Model Garden Deployment with Cloud Deploy

Uniform Bucket-Level Access

Professional Cloud Architect

Choosing Dataproc for Spark and Dataflow for Beam

What Dataproc and Dataflow Are

How This Solution Meets the Requirements

Key Advantages and Capabilities

Why This Is Optimal

Related questions

Cloud Run Scale-to-Zero

Spot VMs Custom Machine Types Autoscaling

BigQuery Omni With Workload Identity Federation

Vertex AI Model Garden Deployment with Cloud Deploy

Uniform Bucket-Level Access

What Dataproc and Dataflow Are

How This Solution Meets the Requirements

Key Advantages and Capabilities

Why This Is Optimal