Prepare and test your skills
Prepare and test your skills
Analyze Logging and Monitoring Infrastructure is the combination of log collection, metric collection, and visualization that shows the health and performance of Google Cloud workloads. Cloud Logging (opens in a new tab) stores, searches, analyzes, and alerts on log data and events from Google Cloud and AWS. Cloud Monitoring (opens in a new tab) collects, stores, visualizes, and alerts on metric data from applications and Google Cloud services. Telemetry flows from resources such as Compute Engine, Google Kubernetes Engine (GKE), Cloud Run, and App Hub applications into these services, where it is charted on dashboards and evaluated by alerting policies.
System telemetry in Google Cloud Observability is log data, metric data, and trace data that together describe applications and their environment. Metric data is numeric health or performance data measured at regular intervals, such as CPU utilization and request latency. Unexpected changes in metrics can signal an issue, while long-term patterns reveal usage trends and resource needs. Log data is timestamped entries where each entry describes a specific event, giving rich detail about one part of an application but not showing how components affect each other.
Trace data connects those isolated events into an end-to-end story. A trace represents the path of a single request across a distributed application and is composed of spans for single functions or operations. Spans show request flow and latency, which supports root-cause analysis when one slow component affects the whole request.
Cloud Logging is the service that collects logging data from application components and Google Cloud services for storage, search, analysis, and alerting. Log entries flow from Compute Engine virtual machines (VMs), GKE clusters, Cloud Run services, and Cloud Run functions into Cloud Logging. Entries can be viewed in Logs Explorer and correlated with metrics and traces on custom dashboards.
Logging also creates signals that flow back into other services. Error Reporting depends on Cloud Logging because it analyzes log entries to find errors, annotates the entries, and creates an error group for investigating cause and history. Log-based metrics record numeric information about logs written to Cloud Logging, including Google-defined counts such as detected errors and total entries received, plus user-defined log-based metrics.
Cloud Monitoring is the service that answers questions such as how much load is on a service, whether a website responds correctly, and whether a service performs well. Cloud Monitoring automatically collects performance information for most Google Cloud services. Additional telemetry flows to it from agents, sidecars, and application instrumentation.
Cloud Monitoring collects and stores the following types of metric data:
Exam tip: Choose user-defined custom metrics when application code must emit a new numeric signal through the API or OpenTelemetry, and choose log-based metrics when the signal can be counted or measured from log entries already written to Cloud Logging.
Visualization and alerting in Cloud Monitoring turn stored telemetry into dashboards, charts, incidents, and synthetic tests. As resources are created or applications are registered with App Hub (opens in a new tab), the dashboard service automatically creates Google Cloud-managed dashboards that show curated logs, metrics, open alerts, and application health. Custom dashboards control exactly what is viewed and can display metric charts and tables, log data and error groups, alerting-policy charts, alert information, text, and system events such as reboots or crashes.
Metrics Explorer visualizes time-series data with comparisons to previous data, outliers, percentiles, and multiple metrics, and saved charts can be added to a custom dashboard. An alerting policy evaluates a metric query and sends notifications through channels including email, Cloud Mobile App, PagerDuty, or Slack when performance meets defined criteria, such as VM CPU utilization exceeding 80%. Each notification links to an incident record that stores status, links to logs, a chart of recorded metric data, labels, and duration. Synthetic monitors provide proactive validation by probing HTTP, HTTPS, and TCP endpoints with uptime checks and by crawling web pages with a broken-link checker.
Default and extended collection determines how much telemetry flows from Compute Engine, GKE, and Cloud Run into Logging and Monitoring. Compute Engine VMs generate only basic system metric and log data by default. GKE clusters send system log and system metric data by default, and Cloud Run generates system metric and log data by default. Cloud Run functions and App Engine automatically generate metric, log, and trace data.
The Ops Agent extends Compute Engine collection by running on the VM to collect host metrics such as CPU, GPU, memory, and process data, system logs such as syslog on Linux and Windows Event Log on Windows, third-party application metrics, Prometheus data, OTLP metric and trace data, and application log data. For containers and serverless workloads, Google Cloud Managed Service for Prometheus handles third-party and user-defined metric collection for GKE, observability metric packages add signals such as control-plane metrics useful for service-level objectives (SLOs), and Cloud Run services that write Prometheus metrics use a Prometheus sidecar to send data to Cloud Monitoring while services that write OTLP metrics use an OpenTelemetry sidecar.
| Service | Primary telemetry | Role in the infrastructure |
|---|---|---|
| Cloud Logging | Log entries and events | Stores, searches, analyzes, and alerts on logs; source for log-based metrics and Error Reporting |
| Cloud Monitoring | Time-series metrics | Collects system, custom, external, Prometheus, and log-based metrics; powers dashboards, charts, SLOs, and alerting policies |
| Cloud Trace | Traces and spans | Follows request flow across components to find latency bottlenecks and shifts in latency profiles |
| Cloud Profiler | CPU and heap profiles | Continuously gathers low-impact production profiles to identify code-level optimization opportunities |
Evaluate Service Reliability and Alerting Strategies is the practice of measuring service performance with targets and responding before failures cascade. Cloud Monitoring (opens in a new tab) connects service-level indicators (SLIs) that measure performance, service-level objectives (SLOs) that state desired performance, and the error budget implied by the SLO to alerting policies and incident records. Teams use the remaining budget to balance release speed against reliability risk.
A left-to-right process flow showing performance metrics forming an SLI, plus goal and period forming an SLO, leaving an error budget monitored by an alerting policy that sends notifications and opens an incident.
A service-level indicator (SLI) is a measurement of how well one aspect of a service is performing. Cloud Monitoring collects underlying performance metrics such as request count, for example HTTP requests per minute that return 2xx or 5xx responses, and response latency, for example latency for HTTP 2xx responses. For known service types including Cloud Service Mesh, Istio on GKE, and App Engine, these metrics are automatically identified. You can also define a custom service type and select its performance metrics.
Standard SLIs are ratios derived from those metrics. An availability SLI is the ratio of successful responses to all responses, while a latency SLI is the ratio of calls below a latency threshold to all calls. Service-specific SLIs can also be defined for other definitions of good performance.
| SLI type | What is counted | Goodness criterion |
|---|---|---|
| Request-based SLI | Atomic units of service, such as successful HTTP requests | Each request either succeeds or fails |
| Windows-based SLI | Time periods, or windows, during which the service behaved well | Performance in the window meets a threshold, such as response latency below a given value |
Exam tip: An availability SLI divides successful responses by all responses, while a latency SLI divides calls below the latency threshold by all calls.
A service-level objective (SLO) is a target value for an SLI measured over a compliance period. An SLO in Cloud Monitoring is built from three parts: the SLI that measures performance, a performance goal that states the desired level, and the compliance period over which the SLI is compared to the goal. You can create up to 500 SLOs for each service in Cloud Monitoring.
SLOs express requirements such as latency exceeding 300 ms in only 5 percent of requests over a rolling 30-day period, or 99% availability over a calendar week. A useful SLO targets less than 100% because the gap between 100% and the target creates the error budget. SLOs are often described as a number of nines, such as 99% or 99.9%, and the highest value you can set is 99.9%.
An error budget is the amount of imperfect performance an SLO allows, starting at 1 minus the SLO and declining as actual performance misses the goal. The error budget for a 99.9% SLO starts at 0.1% of requests or windows. Every failure or slow response consumes part of that budget during the compliance period.
Teams use the remaining budget to decide how fast to change the service. They slow changes when the budget runs low and move faster when budget remains. Changes in SLO compliance and budget consumption can also signal the onset of failures, which is why alerting policies typically monitor SLO compliance and burn rate rather than waiting for the full period to fail.
An alerting policy in Cloud Monitoring defines the metric criteria that trigger a notification and the people or groups who receive it. When the value of a performance metric or SLO burn rate meets the defined threshold, Cloud Monitoring sends notifications through configured notification channels and opens a persistent incident record for troubleshooting. For example, a policy might notify the on-call team when CPU utilization of a VM exceeds 80% or when latency SLI compliance drops too quickly.
Notification channels determine where the alert is delivered for timely response:
Each notification links to an incident, which stores status, duration, labels, a chart of recorded metric data, and links to logs to support diagnosis. The alerting service integrates with many Google Cloud services through pre-configured recommended alerts where you only add the notification recipients. You can create and manage alerting policies by using the Google Cloud console, the Cloud Monitoring API, the Google Cloud CLI, or Terraform.
Utilize Application Performance Management (APM) Tools means using diagnostic services to explain why a distributed system is slow, inefficient, or failing. Cloud Trace (opens in a new tab) adds request-level latency analysis, Cloud Profiler (opens in a new tab) adds code-level performance analysis, and Error Reporting adds automatic error aggregation. Cloud Monitoring and Cloud Logging provide the foundation by collecting metrics and logs that these tools correlate with.
Cloud Trace is a distributed tracing system that collects latency data from applications and displays it in near real time in the Google Cloud console. A single user request flows from the entry-point service through each downstream microservice, and Cloud Trace records how much time the request spends in each span along that path. This end-to-end view pinpoints which service or call in a multi-component pipeline adds latency.
When tracing is enabled, application and access logs carry the same sampling decision, trace ID, and span ID. An engineer can therefore move from a slow trace directly to the related log entries for that request. That link between traces and logs speeds root-cause analysis for slow requests.
Cloud Profiler is a continuous profiling service that identifies performance bottlenecks within function blocks in application code. While Cloud Trace shows time spent between services, Cloud Profiler shows time and resource consumption inside the code itself, such as which functions consume the most CPU or memory. That function-level insight helps teams optimize resource usage and reduce execution time rather than adding larger instances.
Profiler findings connect directly to saturation signals, including CPU, GPU, and TPU utilization, memory usage, and accelerator details such as per-process GPU utilization and temperature from tools like NVIDIA Data Center GPU Manager (DCGM). This connection shows whether costly code is also exhausting critical capacity.
Error Reporting is the service that aggregates and displays errors produced in running applications. Instead of searching individual log streams, Error Reporting groups repeated crashes and exceptions from services on GKE, Cloud Run, App Engine, and other environments into distinct error groups with counts and history. Each group links back to the related stack traces and log context in Cloud Logging.
The team can therefore identify the root cause of application failures and see whether a fix reduced occurrences. Error Reporting answers which failures matter most, while Cloud Trace and Cloud Profiler explain where latency and inefficiency occur.
A complete observability framework combines APM diagnostics with metrics, logs, and SLOs. Cloud Monitoring collects, stores, and visualizes infrastructure and application metrics using pre-built dashboards for Google Cloud services and custom dashboards for workload-specific indicators. Cloud Logging collects detailed logs from applications and underlying infrastructure to provide context around events and errors. Metrics from specialized serving frameworks such as vLLM or NVIDIA Triton Inference Server flow into Cloud Monitoring through Google Cloud Managed Service for Prometheus.
Monitoring centers on the four golden signals that describe user-facing health and capacity:
Infrastructure tracking covers processor usage across CPU, GPU, and TPU, memory usage, network throughput and latency, and disk I/O for both managed environments and self-managed resources like GKE nodes and Cloud Run instances. Alerts on these metrics and on SLO burn rate notify teams before reliability targets are violated.
| Tool | Primary data | Use when |
|---|---|---|
| Cloud Trace | Distributed latency traces across services | A request is slow and the slow service or span must be found |
| Cloud Profiler | Function-level CPU and memory consumption | Code execution is inefficient and the costly function must be found |
| Error Reporting | Aggregated exceptions and crashes | Failures recur and the root cause and frequency must be found |
Exam tip: Choose Cloud Trace for latency across services and Cloud Profiler for resource hotspots inside functions; Trace follows a request between systems, Profiler profiles code within a system.
Cloud Logging stores, searches, analyzes, and alerts on log data and events from Google Cloud and AWS, while Cloud Monitoring collects, stores, visualizes, and alerts on metric data from applications and Google Cloud services. Cloud Logging is the source for log-based metrics and Error Reporting, while Cloud Monitoring powers dashboards, charts, service-level objectives (SLOs), and alerting policies.
Use user-defined custom metrics when application code must emit a new numeric signal through the Cloud Monitoring API or a library such as OpenTelemetry. Use log-based metrics when the signal can be counted or measured from log entries already written to Cloud Logging.
Use Cloud Trace when a request is slow and the slow service or span across distributed services must be found, because it follows request flow and latency between systems. Use Cloud Profiler when code execution is inefficient and the costly function must be found, because it identifies CPU and memory hotspots inside functions within a system.
An e-commerce company runs a microservices-based application across multiple services in Google Cloud. Following a recent feature release, operations engineers notice three distinct issues affecting production:
Which combination of Google Cloud Application Performance Management (APM) and observability tools should you implement to address each issue?
Last updated