Prepare and test your skills

Prepare and test your skills

Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
A financial analytics organization is implementing a CI/CD workflow using Cloud Build to automate testing and quality assurance for their data platform. The codebase includes Apache Beam pipelines intended for Cloud Dataflow, workflow DAGs for Cloud Composer, and schema definitions for BigQuery tables.
The engineering team needs to enforce the following quality assurance lifecycle before code is promoted to production:
DoFn transforms in isolation without provisioning cloud infrastructure.How should the team design their Cloud Build CI/CD pipeline to meet these requirements?
This approach uses the Data Validation Tool (DVT) on production data and relies on a standalone Compute Engine virtual machine to compile and package pipeline code outside of the containerized Cloud Build workflow.
DoFn classes or transform logic.The primary goal was performing post-migration data replication validation between a source database and target BigQuery tables.
This approach executes all pipeline tests on live, managed cloud compute by submitting jobs via the DataflowRunner and running live workflows inside a production Cloud Composer instance.
The requirement was to perform full-scale end-to-end performance and stress load testing on distributed compute before a major version launch.
This configuration bypasses pre-deployment automated testing by immediately deploying artifacts into the production environment and relying on asynchronous runtime monitoring to detect errors after deployment.
DoFn logic.The organization had no CI/CD tooling and relied entirely on manual deployments with post-release operational monitoring.
This solution implements an automated, multi-stage continuous integration and testing pipeline using Cloud Build to validate data pipeline code before deployment. It leverages isolated containerized build steps to sequentially execute static code analysis, Apache Beam unit testing with local runners, integration validation against staging services, and conditional deployment artifact staging.
pytest on Cloud Composer DAG files. This validates Python syntax, catches missing dependencies, and ensures DAGs load without cycles or import errors before any deployment occurs.DoFn transforms and pipeline logic are tested using Beam's built-in TestPipeline with the DirectRunner. This allows for deterministic in-memory execution on synthetic test data without spinning up cloud resources or incurring Dataflow runner costs.TestPipeline with DirectRunner ensures transforms behave as pure functions with predictable inputs and outputs.Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
Keep the momentum going with these hand-picked practice scenarios
Last updated Validated