Prepare and test your skills

Prepare and test your skills

Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An analytics engineering team at an enterprise organization has ingested raw transactional data directly into staging tables in BigQuery. The team consists primarily of SQL developers who need to build a scalable in-warehouse data transformation pipeline.
The solution must satisfy the following technical requirements:
Which Google Cloud service should the team implement?
Dataflow is a fully managed, serverless stream and batch data processing service built on the open-source Apache Beam SDK. It enables execution of complex, parallel data processing pipelines using languages such as Java, Python, and Go.
The team needed to process real-time streaming data from Pub/Sub with complex windowing and aggregations before writing the results into BigQuery, or required heavy programmatic transformations in Python or Java.
Dataform is a fully managed, SQL-based transformation framework in Google Cloud designed to manage, test, version-control, and schedule complex ELT (Extract, Load, Transform) workflows directly inside BigQuery.
.sqlx files) and automatically builds a Directed Acyclic Graph (DAG) of table and view dependencies using standard ref() functions.For SQL-proficient teams executing transformations after data has landed in BigQuery, Dataform provides the most cost-effective, maintainable, and tightly integrated framework compared to spinning up external data processing engines.
Cloud Data Fusion is a fully managed, code-free data integration service built on the open-source CDAP framework. It provides a visual, drag-and-drop web interface for designing ETL/ELT data pipelines and includes over 150 preconfigured connectors.
The organization required a visual, drag-and-drop interface with pre-built connectors to ingest, clean, and transform data from diverse heterogeneous external systems (such as SAP or on-premises databases) without writing code.
Cloud Composer is a fully managed workflow orchestration service built on Apache Airflow. It uses Python-defined Directed Acyclic Graphs (DAGs) to author, schedule, monitor, and coordinate complex multi-cloud and cross-service data pipelines.
The organization needed a global, end-to-end workflow orchestrator to coordinate multi-system processes across multiple cloud platforms, triggering ingestion, machine learning jobs, and external API calls alongside BigQuery jobs.
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
Keep the momentum going with these hand-picked practice scenarios
Last updated Validated