An enterprise data analytics platform processes real-time events across several Google Cloud projects within an organization folder. The data engineering team needs to design a unified observability and alerting architecture using Google Cloud Observability to track pipeline health, custom processing latency, and resource utilization. The solution must satisfy the following technical requirements: Provide a single centralized dashboard in a designated management project that aggregates metrics and logs across all constituent pipeline projects. Ingest custom workload performance metrics and system logs from pipeline worker instances. Trigger automated alerts that notify on-call engineers whenever processing latency anomalies occur, avoiding silent alert executions. Which combination of steps should the team implement?

Configure the management project's metrics scope to include all worker projects and define a default log scope with the appropriate log views; install the Ops Agent on worker instances to stream workload metrics and logs; build a custom dashboard using dashboard configuration JSON; and configure Cloud Monitoring alerting policies linked to explicit notification channels.: ### What the centralized observability architecture is This architecture establishes a centralized observability strategy by consolidating telemetry from multiple Google Cloud projects into a designated management project metrics scope and log scope. It relies on the **Ops Agent** for collecting system and workload telemetry, custom dashboard definitions via configuration files, and **Cloud Monitoring alerting policies** paired with defined notification channels. ### How it meets the requirements - **Aggregates metrics and logs across projects:** Configuring a **metrics scope** in the management project and linking project log views (such as `_Default/_AllLogs`) into a default **log scope** allows engineers to visualize time-series telemetry and logs across all pipeline projects from a single pane of glass. - **Ingests custom workload performance metrics:** Deploying the **Ops Agent** on worker compute resources enables ingestion of system metrics, process logs, and workload-specific metrics (such as latency measurements under `workload.googleapis.com/...`). - **Alerts on-call engineers without silent firing:** Defining **alerting policies** with configured **notification channels** (e.g., PagerDuty, Slack, Email) ensures on-call staff receive proactive incident alerts rather than having policies fire silently without notifying personnel. - **Centralized dashboard:** Deploying dashboards using infrastructure-as-code or `gcloud monitoring dashboards create` with JSON templates creates unified visualization panels for pipeline health. ### Key advantages - **Centralized visibility:** Prevents siloed monitoring across multi-project data processing boundaries. - **Out-of-the-box telemetry:** Ops Agent handles logging and metric streaming seamlessly. - **Operational reliability:** Explicit notification channel binding ensures zero missed degradation incidents. ### Why this is optimal This approach aligns with Google Cloud architecture best practices by decoupling workload execution from centralized monitoring and governance, ensuring complete fidelity without custom log-forwarding infrastructure.

Centralized Pipeline Observability Alerting Monitoring | professional-cloud-data-engineer

Centralized Pipeline Observability Alerting Monitoring

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An enterprise data analytics platform processes real-time events across several Google Cloud projects within an organization folder. The data engineering team needs to design a unified observability and alerting architecture using Google Cloud Observability to track pipeline health, custom processing latency, and resource utilization.

The solution must satisfy the following technical requirements:

Provide a single centralized dashboard in a designated management project that aggregates metrics and logs across all constituent pipeline projects.
Ingest custom workload performance metrics and system logs from pipeline worker instances.
Trigger automated alerts that notify on-call engineers whenever processing latency anomalies occur, avoiding silent alert executions.

Which combination of steps should the team implement?

Export all Cloud Logging entries from each project to Pub/Sub topics; route the messages into an external third-party dashboard tool; and query Cloud Monitoring metrics explorer endpoints via on-demand cron scripts.

Create individual alerting policies in each worker project without notification channels to write alert logs locally; deploy fluentd daemon containers to duplicate metrics to each project's log bucket; and use Metrics Explorer PromQL queries locally within each separate project.

Configure the management project's metrics scope to include all worker projects and define a default log scope with the appropriate log views; install the Ops Agent on worker instances to stream workload metrics and logs; build a custom dashboard using dashboard configuration JSON; and configure Cloud Monitoring alerting policies linked to explicit notification channels.

Disable the Ops Agent to minimize resource utilization on worker instances; rely exclusively on Google-managed hypervisor CPU metrics; and configure BigQuery scheduled queries to scan audit logs every hour for pipeline error alerts.

Professional Cloud Data Engineer

Centralized Pipeline Observability Alerting Monitoring

Related questions