Diagram
A decision tree starting from a data transformation workload, branching on whether the job needs orchestration, existing Spark or Hadoop code, SQL in BigQuery, a visual interface, or continuous parallel processing to recommend Cloud Composer, Dataproc, Dataform, Cloud Data Fusion, or Dataflow respectively.
Matching Each Tool to Its Best-Fit Workload
Selecting a compatible transformation tool means choosing the service whose strengths line up with the clarified requirements. Dataflow is a serverless, graph-based execution system for batch and streaming data processing that scales automatically. Dataproc, the Managed Service for Apache Spark, runs existing Spark and Hadoop workloads without managing the underlying infrastructure details. Cloud Data Fusion provides visual integration for building extract, transform, load (ETL) pipelines, Cloud Composer orchestrates workflows across systems, and Dataform develops SQL-based transformations inside BigQuery.
Domain 1 of 3
29 practice questions
Practice this DomainLast updated