Prepare and test your skills

Prepare and test your skills

Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An enterprise data engineering team is building an event-driven, automated data processing pipeline on Google Cloud. The workflow must execute sequentially whenever new files arrive in Cloud Storage:
The entire orchestration layer must be completely serverless, scale to zero when idle with no persistent compute infrastructure, and provide built-in retry and state-handling capabilities.
Which orchestration approach should the team implement?
Cloud Workflows is a fully managed, serverless orchestration platform that coordinates Google Cloud services, external APIs, and containerized workloads into reliable, automated state machines without requiring any underlying infrastructure management.
googleapis.run.v1.namespaces.jobs.run), Workflows dynamically injects parameters such as input file locations and environment variables into the container via containerOverrides.Unlike heavy, dedicated orchestrators like Apache Airflow or self-managed worker pools, Cloud Workflows paired with Eventarc delivers an entirely serverless, low-latency, and cost-effective solution tailored for coordinating microservices, container jobs, and managed data pipelines.
This approach uses a Compute Engine Managed Instance Group (MIG) running custom software agents that continuously pull messages from Pub/Sub topics and spawn pipeline processes on VM instances using a task-farming architecture.
The team were migrating proprietary, legacy third-party data processing binaries that require fixed VM file systems and lack container or Google Cloud API support.
Dataproc Workflow Templates provide a mechanism to define and execute directed acyclic graphs (DAGs) of Spark, PySpark, MapReduce, Hive, and Pig jobs on ephemeral or managed Dataproc Hadoop/Spark clusters.
OrderedJob specifically on Dataproc.The workflow consisted exclusively of Spark, PySpark, and Hive transformations running on ephemeral Hadoop clusters with structured dependency ordering via start-after flags.
This approach deploys an open-source Apache Airflow instance onto a self-managed Google Kubernetes Engine (GKE) cluster and uses the BashOperator to execute command-line instructions (gcloud run jobs execute and gcloud dataflow jobs run).
BashOperator to invoke gcloud CLI commands bypasses structured status polling, error handling, and parameter typing provided by native service APIs.The scenario required hosting a complex, existing on-premises Airflow migration on container infrastructure with dedicated long-running worker pools and custom Python dependencies.
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
Keep the momentum going with these hand-picked practice scenarios
Last updated Validated