Prepare and test your skills

Prepare and test your skills

Distributed tracing and latency analysis is the practice of following one user request through many services and splitting its total time into timed parts to find delays. Cloud Trace is the Google Cloud distributed tracing system that collects latency data and shows it in near real time in the Google Cloud console. When a request flows from a frontend to a backend to a database or another service, Cloud Trace records the full journey as a trace, and each timed segment helps show which service added the delay.
Cloud Trace works together with logging and metrics as part of Google Cloud Observability. Cloud Logging stores detailed event records for each request, while Cloud Monitoring collects resource and performance data such as CPU use, memory use, and application response times. Cloud Trace connects these views by showing how long a specific request took and how that time was divided across distributed services, so a latency spike on a dashboard can be traced to a slow request. Cloud Run (opens in a new tab) is automatically integrated with Cloud Monitoring with no setup required, which means request latency metrics are available without an agent and the same request can then be examined in Cloud Trace and Cloud Logging.
Scope checking is the first step of latency analysis because it decides whether slowness is widespread or rare. For Cloud Run, open the service or job in the Google Cloud console and click the METRICS tab to show the Request latencies dashboard. The same data can be viewed in Cloud Monitoring by selecting the metric Cloud Run Revision > Request_latencies > Request latency. For App Engine, the latency dashboard shows 50th, 95th, and 99th percentile latency for the service. High values on all three percentiles mean at least 50% of requests are slow, while high latency only in the 99th percentile means about 1% of requests are slow.
The workflow for a slow request moves in order from overall metrics to a single trace and then to a fix. Data flows from the metrics dashboard to the selected trace and then to the scaling or code change, and the workflow depends on correlated logs and traces to move from a spike to the exact request.
A four-step process flow showing how a slow request is diagnosed: check latency scope in Cloud Monitoring metrics, find the slow request in Cloud Logging or Cloud Trace, examine trace spans to pinpoint the bottleneck, then fix it by scaling, optimizing code, or adding indexes.
A trace span is a timed segment that represents one step in request processing, such as handling in the main service or a call to another service. The full trace shows total request time from source to destination, while each span shows where part of that time was spent. When one span is much longer than the others, that span identifies the bottleneck across the distributed services. Dependencies are a common source of long spans because a call from a Cloud Run service to a database, API, or downstream service adds its own network and processing time to the total. Explicit logging for calls to other services makes those dependency spans visible, and without that logging the trace can hide the true source of latency.
Exam tip: A latency spike must be judged against its time window — a large spike in a small window can be negligible in a larger window, so always check the selected time range before concluding that a bottleneck exists.
Correlated logs link a container or application log entry to its request log and to its trace, so an analyst can move from a log line to the full timeline. Cloud Logging and Cloud Trace share this link through trace identifiers stored with each log entry. To find only requests that have trace data, apply the traceSampled=true filter in Cloud Logging to correlate logs in Cloud Logging with traces in Cloud Trace. This correlation flows from the log entry in Cloud Logging to the matching trace in Cloud Trace. Good correlation depends on writing logs in a way that preserves request and trace fields, because those fields enable movement from an error or slow log line to the distributed timeline.
Cold starts and scaling behavior create latency that trace spans attribute to instance startup or waiting for capacity. Increasing the number of minimum instances keeps instances warm so incoming requests do not wait for a new instance to start. Adjusting maximum instances and scaling settings can also limit the number of connections to a backing service, which protects a database or downstream API from overload during traffic growth. For App Engine, similar controls include increasing the instance class, configuring warmup requests to reduce cold-start latency, and adjusting scaling settings for dynamic instances. If spans still show slow code execution after capacity is stable, modifying source code and using Cloud Profiler to find resource-heavy functions is the next step.
Database query performance and indexing is the practice of finding slow database queries and speeding up data retrieval with better indexes. Query Insights is the dashboard that shows which queries create the most database load by execution time, and the index advisor is the feature that recommends new indexes to make those queries faster. Together they move work from detection to resolution in one place, so high load can be traced to specific queries and then fixed with targeted CREATE INDEX commands.
Query Insights is a monitoring tool for Cloud SQL (opens in a new tab) and AlloyDB that detects inefficient queries through database load charts, top queries by load, and active query views. The dashboard lets you select a time range of 1 hour, 6 hours, 1 day, 7 days, 30 days, or a custom range, and you can click and drag on a chart to zoom into a spike where load or latency is high. Visual query plans and application tags help connect a slow query back to its root cause and to the application code or ORM operation that sent it. You do not need to enable AI-assisted troubleshooting to monitor active queries or to obtain index recommendations.
To open the Query Insights dashboard for a Cloud SQL instance, follow this order:
After inefficient queries are visible, the same dashboard leads to index advice and to active query control. A blocked query or a query running much longer than expected can block other dependent queries, and Cloud SQL gives you the option to terminate specific long-running or blocked active queries. For AlloyDB, the Top dimensions by database load table groups load by Queries so the heaviest queries can be opened for recommendation details.
A high database load investigation is an AI-assisted analysis started from the Instance Overview page or the Query Insights dashboard with the Investigate performance button. First you choose a metric such as CPU utilization and a time range, then you zoom into the high-load area, then you click Investigate performance to have Gemini Cloud Assist analyze the period. After about two minutes the Investigation details pane opens with results from comparing recent performance against a baseline.
The Investigation details pane reports three kinds of information:
The time window and baseline for that comparison follow fixed rules. If the selected period is less than 24 hours, Cloud SQL analyzes the entire period, and if it is greater than 24 hours, Cloud SQL analyzes only the last 24 hours of that period. To calculate expected load, Cloud SQL adds 24 hours of baseline data, using the 24 hours previous to the selected period on most days but using the 7th day previous when the selected period occurs on a Monday. During metrics analysis, Cloud SQL checks for significant threshold changes by comparing baseline aggregated data with the analysis window, including but not limited to:
Exam tip: A selection longer than 24 hours analyzes only the last 24 hours, and a Monday selection compares against the 7th day previous instead of the previous 24 hours.
The index advisor is the recommendation engine inside Query Insights that identifies queries that would run faster with a new index. For a selected query, the advisor estimates the benefit and cost of its suggested indexes before anything is created. Selecting a query shows recommendation details that explain the expected result across the workload.
Recommendation details for a query include:
From the same view you can check which other queries share the benefit and then apply the fix. Data flows from the Query Insights recommendation to copied SQL commands to the primary instance where the indexes are created. To create the recommended indexes for an AlloyDB instance, follow this order:
CREATE INDEX ON "public"."demo_order" ("customer_id");Once the indexes exist on the primary instance, future executions of the impacted queries can retrieve rows through the index instead of slower scans, which lowers execution time and improves system responsiveness.
Code-level resource profiling and optimization is the practice of measuring which functions in running code use the most CPU and memory, then fixing that code to run faster and cost less. Cloud Profiler (opens in a new tab) performs this work by continuously collecting production performance data and linking that data back to specific lines of source code. This link lets a team see resource bottlenecks inside the application itself instead of guessing from infrastructure metrics alone.
Cloud Profiler is a statistical, low-overhead profiler that continuously gathers CPU usage and memory-allocation information from production applications. Statistical means Cloud Profiler samples what the application is doing over time rather than recording every single operation, which keeps overhead low enough to run safely in production. The collected data flows from the instrumented application to a Google Cloud project, where Cloud Profiler attributes CPU time and memory allocation to the application's source code. That attribution illuminates performance characteristics of the code and identifies the parts of the application consuming the most resources.
Profilable applications are applications in supported runtimes that have been modified with a Cloud Profiler agent to capture and send data to a Google Cloud project. The profiling agent runs inside the application process, so resource use measured in production is mapped directly to functions in that runtime's code. Applications running outside of Google Cloud can also be profiled when they are configured to send data to a Google Cloud project.
Once the agent is added, Cloud Profiler depends on the running application to keep sending samples, and the Google Cloud project depends on those samples to build readable profiles. Choose the guide that matches the runtime because setup steps and supported profile types differ by language.
Profile views in Cloud Profiler turn continuous CPU and memory samples into visual tools for finding resource-intensive functions. The central view is the flame graph, which stacks function calls so wider blocks represent functions that consumed more of the measured resource. A developer starts with the full profile, then narrows the analysis to confirm which function should be optimized first.
The relationship is direct: the function with the largest share of CPU or memory allocation is the best optimization target because fixing it reduces both response time and resource demand. Over-provisioned resources create unnecessary cost while under-provisioned resources create performance bottlenecks, so reducing waste inside the code helps balance performance and cost. Use Cloud Profiler findings to rewrite, cache, or remove expensive logic before adding larger virtual machines or more instances.
Exam tip: Cloud Profiler is the continuous, low-overhead profiler for CPU and memory use attributed to source code in production, not a one-time load test.
Cloud diagnostics research moves from request behavior to code behavior to data behavior in a connected order. Cloud Trace shows where time is spent as a request flows across services, Cloud Profiler shows which functions consume CPU and memory inside that code, and Query Insights with the index advisor shows which database queries create load and which indexes reduce it. Logs and metrics tie the three views together, so a latency spike leads to a trace, a trace leads to expensive code or a slow query, and the fix is applied to scaling, code, or indexes.
traceSampled=true filter in Cloud Logging finds only log entries that have trace data, linking a slow log line to its full trace in Cloud Trace.Cloud Trace is the Google Cloud distributed tracing system that collects latency data and shows it in near real time in the Google Cloud console. When a request flows from a frontend to a backend to a database or another service, Cloud Trace records the full journey as a trace and divides it into timed trace spans. When one span is much longer than the others, that span identifies the bottleneck across the distributed services.
You can correlate logs with traces by applying the traceSampled=true filter in Cloud Logging to find only requests that have trace data. Cloud Logging and Cloud Trace share the link through trace identifiers stored with each log entry, so the correlation flows from the log entry in Cloud Logging to the matching trace in Cloud Trace. Good correlation depends on writing logs in a way that preserves request and trace fields.
Query Insights is a monitoring tool for Cloud SQL and AlloyDB that detects inefficient queries through database load charts, top queries by load, and active query views. The index advisor is the recommendation engine inside Query Insights that identifies queries that would run faster with a new index and estimates the benefit and cost before anything is created. From the same view you can copy the CREATE INDEX commands and run them on the primary instance so future executions retrieve rows through the index instead of slower scans.
Cloud Profiler is a statistical, low-overhead profiler that continuously gathers CPU usage and memory-allocation information from production applications. Statistical means it samples what the application is doing over time rather than recording every operation, and it attributes CPU time and memory allocation to the application source code. It supports Go, Java, Python, and Node.js applications that have been configured with the Cloud Profiler agent for that runtime, including applications running outside of Google Cloud when configured to send data to a Google Cloud project.
An e-commerce application connected to a Cloud SQL instance is experiencing slow response times during peak business hours. Database CPU utilization is unusually high, and you suspect that unindexed, resource-intensive queries are degrading overall database performance.
How should you examine the database performance using Google Cloud diagnostic tools and implement recommended index improvements?
Last updated