Diagram
A process flow comparing two data patterns: ETL extracts from a source, transforms with Dataflow, Data Fusion, Spark, or Dataprep, then loads into BigQuery, while ELT loads raw data into a BigQuery staging table first and then transforms it with SQL into the production table.
Extract, Transform, and Load (ETL)
Extract, transform, and load (ETL) is the pattern where data is extracted from the source, transformed in an intermediate processing service, and only then loaded into BigQuery as cleaned production data. A file-based ETL flow moves data from the source system to staging, then copies that data to a Cloud Storage bucket, then reads from the Cloud Storage bucket into a transformation service that writes the result to BigQuery. A JDBC-based ETL flow reads directly from the source database through a JDBC driver into the transformation service, which then writes the transformed rows to BigQuery. Choose ETL when data must be trimmed, cleansed, or enriched before loading, or when an existing transformation process must be preserved.
Domain 4 of 7
24 practice questions
Practice this DomainLast updated