Diagram
A left-to-right pipeline showing producers sending events to Pub/Sub, which delivers messages to Dataflow for transformation; Dataflow writes results to Cloud Storage, from where loads and federated queries feed BigQuery and stored batches feed a Dataproc Spark job whose output also reaches BigQuery, all governed by Dataplex for metadata, quality, and lineage.
Choosing the Right Tool
The services compose into one pipeline: Pub/Sub ingests events and passes them to Dataflow, Dataflow transforms and writes results to storage, Dataproc can process stored batches with Spark, and Dataplex governs the datasets that all three produce.
Domain 1 of 6
26 practice questions
Practice this DomainLast updated