Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Highly Resilient Private IP Cloud Composer
Free sample question, answered and explained below. The Professional Data Engineer practice bank has 1,353 more like it. Try 10 free: instant score, no account.
An enterprise is designing a centralized orchestration solution for complex batch ETL/ELT pipelines spanning multiple Google Cloud services and on-premises data sources. The architecture must satisfy the following operational, security, and resilience requirements:
- Workflows must be authored and maintained natively in Python as Directed Acyclic Graphs (DAGs) with automated dependency resolution.
- Database connection strings, API keys, and sensitive service credentials must be stored securely and retrieved dynamically without hardcoding them in pipeline files.
- The orchestration infrastructure must be protected against single-zone failures through multi-zone scheduler redundancy and automated database failover.
- Service access must adhere to the principle of least privilege using a custom identity rather than overly broad default accounts.
Which architecture and configuration should the organization deploy?
Explanations
Deploy Cloud Composer in a Highly Resilient Private IP configuration using a dedicated user-managed service account granted the Composer Worker role, and configure Secret Manager as the Airflow backend for connections and variables.
Deploy Cloud Composer in a Highly Resilient Private IP configuration using a dedicated user-managed service account granted the Composer Worker role, and configure Secret Manager as the Airflow backend for connections and variables.
What Cloud Composer in a Highly Resilient Configuration Is
Cloud Composer is Google Cloud's managed workflow orchestration service built on Apache Airflow. In a Highly Resilient Private IP configuration, Cloud Composer deploys Airflow components across multiple availability zones with automated failover capabilities and private network isolation.
How It Meets the Requirements
- Python DAG Authoring: Cloud Composer natively manages workflows defined in Python as Directed Acyclic Graphs (DAGs), enabling automatic dependency resolution and seamless integration with over 150 Google Cloud operators.
- Secret Manager Integration: Cloud Composer supports configuring Secret Manager as the backend for Airflow connections and variables, allowing workers to fetch sensitive credentials dynamically without storing secrets in DAG code or plain environment variables.
- Operational Resilience: The Highly Resilient architecture provides a regional Cloud SQL metadata database with primary and standby instances distributed across zones, alongside redundant Airflow schedulers, DAG processors, and web servers running in separate zones.
- Least-Privilege IAM: Assigning a dedicated user-managed service account with the Composer Worker (
composer.worker) role guarantees that Airflow tasks only execute with the exact permissions assigned, preventing the security risks associated with default accounts.
Key Advantages and Capabilities
- Multi-Zone High Availability: Automatically recovers from zonal infrastructure failures without dropping pipeline scheduling or corrupting metadata state.
- Centralized Telemetry: Streams component and DAG execution logs directly to Cloud Logging and metrics to Cloud Monitoring for immediate observability.
- VPC Service Controls Compatibility: Private IP environments can be placed inside security perimeters to mitigate data exfiltration risks.
Why This Is Optimal
This architecture directly fulfills all requirements for Python-native data orchestration, enterprise secret management, high availability, and least-privilege access control without requiring custom infrastructure management.
Deploy Cloud Workflows using YAML definitions, inject sensitive connection strings via static workflow environment variables, and grant the workflow's service account the Editor role to access target data services.
Deploy Cloud Workflows using YAML definitions, inject sensitive connection strings via static workflow environment variables, and grant the workflow's service account the Editor role to access target data services.
What Cloud Workflows Is
Google Cloud Workflows is a fully managed, serverless orchestration engine designed to link and coordinate HTTP-based microservices and Google Cloud APIs using imperative workflows defined in YAML or JSON.
Why It Doesn't Meet the Requirements
- Python-Native DAGs: Cloud Workflows requires definitions written in YAML or JSON syntax rather than declarative Python DAGs with built-in Airflow operators and dependency management.
- Secure Secret Handling: Storing sensitive credentials in static workflow environment variables exposes secrets in plain text within configuration files and execution metadata, violating secure credential management standards.
- Least-Privilege IAM: Assigning the primitive Editor role grants broad, unnecessary permissions across the entire Google Cloud project rather than scoped access to specific target services.
Key Characteristics and Limitations
- Optimized for low-latency, high-volume HTTP service orchestration and microservice chaining rather than heavy, batch ETL/ELT pipelines.
- Variables passed within execution state are constrained by memory quotas (512 KB limit for execution memory).
- Lacks the native data ecosystem connectors and complex data backfill capabilities found in Apache Airflow.
This Answer Would Be Correct If
The organization required a lightweight, serverless orchestrator to trigger cloud functions or HTTP APIs with sub-second execution overhead and YAML-based workflow logic.
Deploy Cloud Composer in a standard single-zone Public IP configuration using the default Compute Engine service account, and store Airflow connections inside the Airflow metadata database.
Deploy Cloud Composer in a standard single-zone Public IP configuration using the default Compute Engine service account, and store Airflow connections inside the Airflow metadata database.
What a Standard Single-Zone Cloud Composer Configuration Is
A standard Public IP Cloud Composer deployment provisions Airflow components within a single compute zone and relies on standard metadata storage located within the tenant project.
Why It Doesn't Meet the Requirements
- Operational Resilience: A single-zone deployment lacks metadata database failover and redundant schedulers across zones. If the hosting zone experiences an outage, Airflow schedulers and tasks are disrupted.
- Least-Privilege IAM: Relying on the default Compute Engine service account introduces severe security risks because it frequently carries the basic Editor role, exposing excessive privileges to anyone authoring DAGs.
- Secret Management: Storing sensitive connection details directly inside the local Airflow metadata database without external backend integration increases operational complexity and vulnerability to privilege escalation.
Key Characteristics and Limitations
- Single point of failure for the Airflow metadata database and scheduler instances.
- Elevated security risk because any user with bucket write access can execute code inheriting full Editor permissions.
- Lacks the automated secret rotation and externalized credential auditing provided by Secret Manager.
This Answer Would Be Correct If
The project were a non-critical development or testing environment where single-zone availability was acceptable and security requirements did not mandate private networking or strict least privilege.
Deploy Cloud Workflows to orchestrate containerized Python tasks on Google Kubernetes Engine Autopilot, using Kubernetes Secrets for credential storage and standard cluster auto-repair for resilience.
Deploy Cloud Workflows to orchestrate containerized Python tasks on Google Kubernetes Engine Autopilot, using Kubernetes Secrets for credential storage and standard cluster auto-repair for resilience.
What a Workflows and GKE Hybrid Architecture Is
This pattern uses Cloud Workflows to make HTTP calls to trigger containerized batch jobs or pods hosted on a Google Kubernetes Engine (GKE) Autopilot cluster, leveraging Kubernetes Secrets to inject credentials.
Why It Doesn't Meet the Requirements
- Native Python DAG Management: The orchestration layer itself is managed in YAML/JSON via Cloud Workflows rather than through a native Python DAG orchestration framework with built-in task lineage and dependency scheduling.
- Operational Overhead: Managing custom container builds, Kubernetes pod manifests, and API integrations across separate services introduces substantial architectural complexity compared to a managed Airflow environment.
- Secret Management: Storing credentials in Kubernetes Secrets requires managing secret synchronization within the cluster rather than using a native Google Cloud Secret Manager backend integration for connections.
Key Characteristics and Limitations
- Requires separate lifecycle management and CI/CD pipelines for GKE pod containers, Kubernetes manifests, and Workflows definitions.
- Lacks unified Airflow UI features such as task instance clearing, interactive backfilling, and visual graph dependency rendering.
This Answer Would Be Correct If
The workload consisted of decoupled, microservice-based container tasks running arbitrary custom runtimes that only needed basic HTTP scheduling rather than complex ETL dependency trees.
Try 10 interactive questions like this one, free
Answer them yourself and get your score the moment you finish. No account needed.
Full access includes
- 1,353 practice questions across all 5 exam domains
- Every answer explained: why it is right, and why the others are not
- Scout, an AI tutor that answers your follow-up questions
- Readiness tracking by domain, so you know when to book the exam
Related questions
Keep the momentum going with these hand-picked practice scenarios
AdvancedProfessional Data Engineer
Cloud Composer Dataproc Ephemeral Cluster Orchestration
Job automation and orchestration (e.g., Cloud Composer and Workflows)
AdvancedProfessional Data Engineer
Eventarc to Cloud Workflows
Job automation and orchestration (e.g., Cloud Composer and Workflows)
AdvancedProfessional Data Engineer
BigQuery vs Dataproc for Interactive and Batch Workloads
Interactive or batch query jobs
AdvancedProfessional Data Engineer
Looker Studio Service Account Credentials
Publishing reports and visualizations
AdvancedProfessional Data Engineer
Cloud Spanner INSERT_OR_UPDATE Idempotent Writes
Defining data transformation and orchestration logic
FundamentalAssociate Data Practitioner
Cloud SQL Cross-Region Replication Lag
Determine when to use replication
Last updated Validated