Analyzing Logs with Logs Explorer Queries

Advanced queries in Cloud Logging are Boolean expressions that select a subset of log entries from a project, folder, or organization. The same Logging query language (opens in a new tab) runs in Logs Explorer (opens in a new tab), the Logging API entries.list method, the gcloud logging read command, and in filters for sinks and log-based metrics. Every query is built from LogEntry indexed fields such as resource.type, severity, logName, timestamp, textPayload, jsonPayload, and protoPayload.

1.1Query Structure and Logical Operators

Query structure defines how comparisons combine into a complete filter expression. A comparison is either a global value or a [FIELD_NAME] [OP] [VALUE] test, joined by the rule expression = ["NOT"] comparison { ("AND" | "OR") ["NOT"] comparison }. A simple restriction such as resource.type = "k8s_cluster" tests one field, a conjunctive restriction such as resource.type = "k8s_cluster" AND severity = "ERROR" requires both tests to match, and a disjunctive restriction such as resource.type = ("k8s_cluster" OR "gce_instance") matches either value. Complex expressions use parentheses to control order, for example resource.type = "k8s_cluster" AND (severity = "ERROR" OR "error").

Separate lines in the query editor join with an implicit AND when no operator is written, so resource.type = "gce_instance" on one line and severity >= "ERROR" on the next line evaluates as resource.type = "gce_instance" AND severity >= "ERROR". Logical operators AND, OR, and NOT must be capitalized, while field names and most string comparisons are not case sensitive. Queries cannot exceed 20,000 characters, and regular expressions remain case sensitive even though the rest of the language is case-insensitive.

1.2Filtering by Resource Type, Severity, and Log Name

Resource, severity, and log-name filters narrow a query to the exact source that matters for root-cause analysis. The fields resource.type, resource.labels.instance_id, severity, and logName identify where a log entry came from and how serious it is. For example, resource.type = "gce_instance" AND severity >= "ERROR" AND NOT textPayload:robot matches Compute Engine entries with severity of at least ERROR whose textPayload field does not contain the string robot anywhere inside it. Severity comparisons support ordering, so >= "ERROR" includes ERROR and higher severities.

Log names must use the full encoded path because the short display name does not match. The Admin Activity audit log is named cloudaudit.googleapis.com/activity, so the correct test is logName = "projects/my-project-id/logs/cloudaudit.googleapis.com%2Factivity" with %2F for the slash. A precise source filter then combines that log name with nested resource labels, for example resource.type = "gce_instance" AND resource.labels.instance_id = "6731710280662790612" plus the encoded logName.

1.3Filtering by Payload Content With Nested Field Paths

Payload filtering uses dotted paths into structured log data to test message text and event fields. The path textPayload tests an unstructured text message, jsonPayload.my_favorite_cat tests a single field inside a JSON payload, and protoPayload.metadata.event.eventName tests an audit event name alongside resource.type = "audited_resource". For Google Workspace login auditing, protoPayload.metadata.event.eventName = "password_edit" with resource.type = "audited_resource" isolates password-change events. Checking one entry first reveals the correct label and payload names before those paths are used in a broader query.

The match operator controls whether the test is exact or partial. The following comparison shows three supported matching styles across shared behavior and use:

A bare quoted value such as "Hello Kitty" is a global restriction that implicitly uses the has operator against every field and payload, so it is the slowest form. Limiting the same text to one field, using equality when possible, or referencing a structured field such as jsonPayload.my_favorite_cat = "Hello Kitty" keeps the intent while reducing the data scanned.

1.4Isolating Historical Events With Time Ranges

Time-range filtering uses the timestamp field with RFC 3339 values to isolate a failure window during root-cause analysis. A lower bound such as timestamp >= "2023-06-27T14:40:00-04:00" selects entries from that point forward, while a bounded range such as timestamp >= "2023-06-27T12:40:00-04:00" AND timestamp <= "2023-06-27T14:40:00-04:00" selects only entries between those two times. Timestamps must use the letter T between date and time, so a value from the GNU/Linux date --rfc-3339=s command is converted from 2023-06-27 14:40:00-04:00 to 2023-06-27T14:40:00-04:00. Combining that timestamp test with resource and severity tests focuses the result set on the affected system during the incident period.

1.5Writing Fast, Index-Friendly Queries

Index-friendly queries prefer conjunctive AND tests on indexed fields and avoid global or substring searches. Queries that use only AND, such as SEARCH("foo") AND SEARCH("bar"), can make better use of indexes and run faster than OR queries such as SEARCH("foo") OR SEARCH("bar"), which are slow and cannot use indexes as effectively. When OR is required, grouping the clauses with parentheses preserves the intended logic.

Precise field restrictions reduce cost and noise compared with shortcuts:

• Limit a payload search to one field with textPayload:"Hello Kitty" instead of a global "Hello Kitty" search.
• Prefer equality with textPayload = "Hello Kitty" when the complete value is known.
• Reference a structured field with jsonPayload.my_favorite_cat = "Hello Kitty" when entries have JSON payloads.
• Add an indexed restriction such as logName = "projects/my-project_id/logs/somelog" before the payload test.
• Use the SEARCH function with complete text and avoid partial text that will not match.

These patterns keep data flowing from the selected log bucket through the filter to the query results with fewer false matches, which makes repeated troubleshooting queries faster and more reliable.

Logs Explorer is the Google Cloud console interface for displaying, parsing, and analyzing log entries stored in log buckets. The Logs Explorer (opens in a new tab) interface combines query controls with diagnostic panes that reveal volume patterns, field distributions, severity signals, and parent-child relationships across a request flow.

2.1Logs Explorer Interface and Diagnostic Panes

The Logs Explorer interface organizes diagnostic tools into five sections that work together for troubleshooting. The Query pane builds filters, the Fields pane surfaces field values and patterns, the Timeline visualizes entries across the selected time range, the Query results pane lists matching entries, and the primary toolbar controls time range, streaming, sharing, and display preferences. Together these panes let an operator move from a broad distribution anomaly to specific correlated entries without leaving the page.

The primary toolbar manages the scope and presentation of the analysis:

• Query library: view recent, saved, suggested, and Google Cloud-provided queries for common use cases.
• Share link: create a shortened URL that preserves the absolute time range of the query, such as 7:49:37 PM - 8:49:37 PM.
• Preferences: show or hide the Timeline and Fields panes, hide summary chips, change sorting order, configure the Time column, wrap lines, and manage summary fields.
• Time-range selector: specify the time window searched for log entries.
• Stream logs: view log entries as Logging stores them in log buckets.
• Learn: view links to relevant documentation.

Use Preferences to adapt the layout to the failure being investigated. Select View to show or hide the Timeline and Fields panes or to change sort order, select Format to configure the Time column or to wrap up to 10 lines per entry instead of truncating to one line, and select Manage summary fields to surface repeating patterns.

2.2Timeline and Distribution Analysis

The Timeline pane in Logs Explorer shows how matching log entries distribute across the time range selected in the time-range selector. A sudden spike, gap, or cluster in the Timeline signals a change in system behavior that warrants drilling into the Query results pane for that interval. Because the Timeline, query filter, and time-range selector stay synchronized, narrowing the time range or adding a filter immediately reshapes the distribution view.

Chronological correlation starts with time scoping. The page searches log entries that originate in the selected project, folder, or organization, including entries routed to the project by a sink in another project and then stored in a log bucket. If an expanded parent shows no children, no other log entries within the query's time range share that trace ID, so widening the time range or removing an overly restrictive filter is the next diagnostic step.

2.3Fields Pane and Severity Signals

The Fields pane in Logs Explorer parses log entry fields so anomalous values and high-cardinality dimensions can be spotted without reading every payload. Summary fields extend this capability by highlighting repeating patterns across the result set, which helps isolate a failing version, route, method, or resource type during a cascading failure. Entering a search term or updating the query expression further filters the displayed entries, for example showing only entries that contain "app log message".

Severity provides the second triage signal alongside distribution. The severity field in a LogEntry supports comparisons such as severity >= "ERROR", and conjunctive filters such as resource.type = "gce_instance" AND severity >= "ERROR" isolate error and critical entries from a single resource type. String comparisons are not case sensitive, while logical operators AND, OR, and NOT must be capitalized and queries cannot exceed 20,000 characters.

2.4Correlate Log Entries by Trace

Correlation in Logs Explorer groups log entries that share the same trace field into a parent-child view, which simplifies tracing a single request across distributed microservices. When correlation is enabled, the Query results pane displays one row per parent entry, and expanding a parent with chevron_right Expand this log entry nests its child entries underneath it in chronological and contextual order.

Trace context originates from instrumentation. When an application is instrumented with OpenTelemetry (opens in a new tab), trace and span IDs are extracted from the trace context and included in the OTLP log data sent to the project, and Google Cloud exporters also extract these fields. The trace field can also be set manually. Trace-aware log entries are written natively by App Engine standard environment, Cloud Run functions, and Cloud Run, while Compute Engine and Google Kubernetes Engine include trace data when they use the Ops Agent or the OpenTelemetry Collector.

For Logs Explorer to correlate a collection of log entries, all of the following requirements must hold:

• The trace field is set to TRACE_ID or to projects/PROJECT_ID/traces/TRACE_ID, where PROJECT_ID is the project ID and TRACE_ID is the trace ID.
• The value of the trace field is the same across all log entries in the group.
• Each log entry has a different logName value.
• The timestamp of the parent log entry is no later than the timestamp of any child log entry.

To view and search correlated log entries (opens in a new tab) in Logs Explorer, follow this order:

1. Open Logs Explorer and select Correlate by: Go to the Logs Explorer page and, in the Correlate by menu, select the parent log name, for example request_log for App Engine applications.
2. Review parent rows and expand: The Query results pane displays one row for each entry matching the parent log name; click Expand this log entry to reveal nested children that share the trace ID within the query time range.
3. Filter the correlated set: Enter a search term or update the query expression and click Run query to show only correlated entries that contain the term.

Converting log queries into actionable metrics and reusable assets means turning a one-time troubleshooting filter into a saved query or a continuously evaluated metric. In Cloud Logging, the same Logging query language expression runs in the Logs Explorer, the Logging API, and the Google Cloud CLI, so a proven filter can be reused without rewriting. When counting, grouping, or trending is needed, the workflow moves from the Logs Explorer to Observability Analytics or to a log-based metric that feeds Cloud Monitoring.

3.1Reusable Queries Across Logging Interfaces

Reusable queries in Cloud Logging (opens in a new tab) are Logging query language expressions that return the same matching log entries wherever they run. The same query language is supported by the Cloud Logging API, the Google Cloud CLI, and the Logs Explorer, and queries can be issued and saved through the Logging API commands. In the Logs Explorer, queries can be built by using menus, by entering text, and in some cases by using options included with the display of an individual log entry. Because the language is shared, a filter validated during troubleshooting in the console can later run programmatically through gcloud logging read or entries.list without changing its logic.

3.2Choosing Between Logs Explorer and Observability Analytics

The choice between Logs Explorer and Observability Analytics depends on whether the task is inspecting individual entries or generating grouped insights and trends. Logs Explorer is designed to troubleshoot and analyze performance of services and applications, with a histogram that shows the rate of errors and annotations that link a log entry to its error group. Observability Analytics (opens in a new tab) is designed to group and aggregate log data, join log and trace data, and display results as a table, a chart, or both.

Logs Explorer does not support aggregate operations, like counting the number of log entries that contain a specific pattern. To perform aggregate operations, enable analytics on the log bucket and then use Observability Analytics.

Observability Analytics queries operate against upgraded log buckets through specific views and datasets:

• Grouping and aggregating log data, such as grouping HTTP requests by hour and computing average latency for a URL, flows from upgraded log buckets to table and chart results.
• SQL queries that use pipe syntax run against log data to produce grouped insights and trends.
• Queries of log views and analytics views run against stored logs, where log views use a system-defined schema and analytics views use a user-defined schema.
• Joins of log and trace data combine both signals in one analysis, and linked BigQuery datasets allow BigQuery services to query log data without exporting it.

Upgrading a log bucket does not affect use of the Logs Explorer (opens in a new tab), which only requires that log data be stored in a log bucket. Log entries written before a bucket is upgraded are not immediately available in Observability Analytics, but become analyzable when the backfill operation completes, a process that might take several days.

3.3Creating Log-Based Metrics for Monitoring and Alerting

Log-based metrics are custom metrics derived from matching log entries that track system behavior over time in Cloud Monitoring. A log-based metric applies a Logging query language filter to the stream of incoming entries, so every new entry that matches the filter increments or contributes to the metric time series. The metric time series then flows from Cloud Logging to Cloud Monitoring, where it can be charted for performance trends and attached to an alerting policy. For example, a counter log-based metric can capture accesses of sensitive content, and an alerting policy can notify when the number of accesses in a given period exceeds a specified threshold, giving timely awareness of issues surfaced by logs.

3.4Moving From Logs Explorer Results to Observability Analytics

The Analyze results path is the handoff that converts a Logs Explorer result set into an Observability Analytics SQL analysis. In the Query results toolbar, clicking Analyze results opens the Observability Analytics page in a new tab and automatically translates the current Logging query language query into a SQL query. The button is visible only when the log bucket behind at least one selected log view has been upgraded to use Observability Analytics, and the time range may be adjusted automatically so meaningful data appears. In Observability Analytics, results can be viewed in tabular form or as a chart, and a chart can be customized and saved to a custom dashboard.

The resource selected by the project picker and the toolbar setting determine how the FROM clause in the generated SQL query is constructed:

1. Project logs or _Default scope: The resource's _Required and _Default buckets are queried for matching entries.
2. N log view scope: Log views on log buckets in the current project are queried, while log views on buckets stored by different resources are ignored.
3. Custom log scope: The same project and log-view rules are applied to the resources listed in the selected log scope.

Observability Analytics does not support querying across projects or regions, so if the Logs Explorer searched multiple projects or regions, the Observability Analytics page might show different entries. An upgraded bucket must have been created at the Google Cloud project level, must be unlocked unless it is the _Required bucket, and must have no pending updates. The Observability Analytics page cannot query log views when the bucket has field-level access controls configured, but those entries remain queryable through the Logs Explorer page and through a linked BigQuery dataset.

The full workflow flows from precise Logging query language filters through Logs Explorer diagnosis to durable operational assets. Filtered entries reveal distribution spikes in the Timeline, field patterns in the Fields pane, and request flows through trace correlation, and the validated query then becomes a saved query, an Observability Analytics SQL analysis for aggregation, or a log-based metric that feeds Cloud Monitoring charts and alerts.

Key Facts

• Logging query language queries cannot exceed 20,000 characters, require capitalized AND, OR, and NOT, and treat field names as case-insensitive while regular expressions remain case sensitive.
• The Admin Activity audit log must be filtered as logName = "projects/my-project-id/logs/cloudaudit.googleapis.com%2Factivity" with %2F URL encoding.
• Logs Explorer correlation requires the same trace value across entries, a different logName for each entry, and a parent timestamp no later than any child timestamp.
• Logs Explorer finds and views individual log entries but cannot count or aggregate them; aggregation requires enabling analytics on the log bucket and using Observability Analytics with SQL.
• A log-based metric applies a Logging query language filter to incoming entries and sends the resulting time series from Cloud Logging to Cloud Monitoring for charting and alerting.