Diagram
Events from source apps and operational databases flow into a Pub/Sub topic, through a Dataflow Beam pipeline for filtering and transformation, into BigQuery tables for near real-time SQL analytics.
Dataflow for Unified Stream and Batch Processing
A common streaming pattern connects messaging to warehousing through Dataflow in order:
The same pattern works with change data: Datastream changes can flow through Dataflow job templates into BigQuery, and MySQL changes can be continuously replicated to a BigQuery table. Choose Dataflow when data must be cleaned, joined, or reshaped between its source and BigQuery.
Domain 1 of 4
24 practice questions
Practice this DomainLast updated