Audit Logs Analysis and Security Alerting

Log-based monitoring for security events turns Cloud Audit Logs (opens in a new tab) entries into measurable signals and timely notifications. In this design, Cloud Logging collects and filters log entries, Cloud Monitoring evaluates alerting conditions, and notification channels deliver findings to responders and external systems. The goal is to record who did what, where, and when, then separate events that need immediate investigation from low-priority noise.

1.1Always-On Audit Foundation for Alerting

The reliable foundation for security alerting is Admin Activity audit logs, which are always written and cannot be configured, excluded, or disabled. These logs are still generated even if the Cloud Logging API is disabled, so alerts built on administrator activity do not go silent because of a configuration change. By contrast, Data Access audit logs must be explicitly enabled, can be very large, and can increase storage cost, so they are often excluded in development projects and enabled selectively in production. This creates a core tradeoff where Admin Activity alerts stay broad and mandatory while Data Access alerts stay narrowly scoped to high-value reads and exports.

1.2Centralizing Identity and Login Signals

Centralized monitoring depends on bringing identity events into the same place as resource events. Admin Activity audit logs from Cloud Identity and Google Workspace are ordinarily managed and viewed in the Google Admin console, separately from Google Cloud logs, but they contain user login events that are relevant to Google Cloud. Sharing Cloud Identity audit logs to the Google Cloud environment lets one set of filters and alerts cover both logins and resource changes. Once centralized, entries can be found in Logs Explorer by selecting the organization, choosing a resource type, and choosing a log name of activity, data_access, system_event, or policy.

1.3Creating Log-Based Metrics and Alerting Policies

A log-based monitoring pipeline moves data from log entries to human action in a fixed order. The pipeline depends on Cloud Logging to select events and on Cloud Monitoring to decide when to notify:

1. Filter log entries: Cloud Logging evaluates a filter against incoming audit log entries and counts matches as a log-based metric or evaluates a log-based alerting condition.
2. Evaluate alerting policy: Cloud Monitoring evaluates the alerting policy against that signal and fires an incident when the described condition occurs.
3. Deliver incident: The incident flows from the alerting policy to notification channels such as a monitored group email, Pub/Sub topic, or custom dashboard in Cloud Monitoring.

Policies should distinguish events that require immediate investigation from low-priority events rather than alerting equally on every match. For example, a policy can match a specific data-access message for an anomalous export and notify on each appearance, while a separate policy aggregates routine reads for dashboard review. Custom dashboards in Cloud Monitoring then track critical metrics over time, so teams see trends in failed logins, privilege changes, and data access alongside active alerts.

1.4High-Value Security Events to Alert On

Critical security monitoring maps specific audit-log changes to attacker techniques and insider-risk patterns. Dedicated log metric filters paired with alerts should cover these patterns:

• Unauthorized compute changes such as unexpected virtual machine creations that can signal hijacked credentials or cryptomining setup.
• Identity privilege escalation such as IAM policy grants, project ownership assignments or changes, and grants of Service Account User or Service Account Token Creator roles.
• Audit and role tampering such as Audit Configuration changes and custom IAM role changes that can blind future detection.
• Network exposure changes such as VPC network firewall rule creations, edits, or deletions that open sensitive ports.
• Authentication abuse such as excessive failed logins from Cloud Identity logs and use of user-managed service account keys outside expected patterns.
• Data exfiltration and misuse signals such as anomalous Data Access reads and exports, additions of non-corporate accounts to privileged groups, and sudden billing spikes that suggest unauthorized resource use.

When any of these signals fires outside typical or expected patterns, the defined review process moves from detection to timely containment, privilege reversal, and audit-record preservation. Billing anomaly detection and budget alerts provide an independent early warning when log coverage is incomplete, because unexpected spend often appears when an attacker provisions resources.

1.5Routing Findings for Retention and SIEM Integration

Long-term detection and investigation depend on routing log data beyond the default log buckets. Using sinks, Cloud Logging routes selected logs to supported destinations including Cloud Storage for long-term preservation, BigQuery for detailed analysis, and Pub/Sub for streaming integration. Traffic flows from the sink to the Pub/Sub topic, and from the Pub/Sub subscription to a security information and event management (SIEM) tool or to Google Security Operations (opens in a new tab) for correlation. Security bulletin notifications and budget programmatic notifications can use the same Pub/Sub delivery pattern with filtering for the notification types to receive.

Retention and delivery reliability are controlled at the storage and contact layers. Cloud Logging retains logs in a log bucket according to custom retention rules from 1 day to 3650 days, with Bucket Lock used to enforce retention policies on log buckets so audit evidence cannot be deleted early. Critical security alerts flow from Cloud Monitoring to Essential Contacts configured as a monitored group alias or mailing list, so a compromise or account alert reaches an active team rather than an individual who may be unavailable.

Analysis of Data Access logs for anomaly detection and forensics is the practice of using Cloud Audit Logs (opens in a new tab) to answer who did what, where, and when across Google Cloud resources. Cloud Logging collects immutable log entries from Google Cloud services, and investigators correlate entries across services and time to find unauthorized access, misuse of permissions, and the sequence of events in a security incident. Data Access logs are central to this work because they record reads and writes to user data and configuration that Admin Activity logs do not cover.

2.1Distinguishing Audit Log Types for Investigations

Audit log types in Cloud Audit Logs differ by what triggers them and whether they must be enabled. Admin Activity logs record user-driven API calls that modify configuration or metadata, Data Access logs record reads of configuration and reads or writes of user-provided data, Policy Denied logs record denied access due to a security policy violation, and System Event logs record changes made by Google systems without user action. For identity and access investigations, Admin Activity logs and Policy Denied logs are the recommended starting point, with Data Access logs added to reveal data-level access patterns.

A single Cloud Storage request can generate more than one audit entry because entries are generated from the IAM permissions checked to complete the request. Cloud Storage operations therefore map approximately, not one-to-one, to Admin Activity, ADMIN_READ, DATA_READ, and DATA_WRITE entries.

2.2Reading Identity, Resource, and Time Fields to Reconstruct Events

An audit log entry for forensics establishes the caller, the target, the action, and the time. Cloud Audit Logs stores the action taken by the user, where the activity occurred, and when it occurred, which lets an investigator link a principal to a method on a specific resource at a specific timestamp. Caller identity, caller IP address, resource type and labels, and protoPayload.methodName together show which account or service account invoked which API against which project, service account, or data object.

Correlation depends on pivoting across those fields over a timeframe. An investigator first fixes the time window around a suspected breach or IAM change, then filters by logName, such as projects/<PROJECT>/logs/cloudaudit.googleapis.com%2Factivity, by resource.type such as project or service_account, and by protoPayload.methodName such as SetIamPolicy. Because log entries are immutable, the resulting sequence can be trusted as a record of what was attempted and changed, including whether the entry came from Admin Activity, Data Access, or Policy Denied streams.

2.3Analyzing IAM Permission Usage and Unauthorized Access Patterns

Analysis of IAM permission usage focuses on lifecycle events that grant, expand, or persist access. Admin Activity audit logs store creation of a service account, changes to IAM policy, and download or creation of a service account key, all of which are high-value signals for privilege escalation and persistence. Policy Denied audit logs complement them by recording when a user or service account is denied access to a Google Cloud service due to a security policy violation, which often reveals probing, excessive permissions, or controls working as intended.

Anomalies emerge when those signals deviate from expected patterns:

• A SetIamPolicy call that grants administrative roles outside a change window or from an unusual principal or IP address.
• Creation of a new service account followed quickly by creation of a service account key and data reads using that account.
• Repeated Policy Denied entries for the same principal across multiple services, followed by a successful Admin Activity change.
• Changes to Google group membership that expand IAM policy membership to non-corporate accounts when groups are used to assign policies.

Investigators use these patterns for two priority scenarios: analyzing permissions and access to resources during a data breach, and analyzing production issues caused by an IAM policy change to verify which user or process made the change. Sharing audit logs from Cloud Identity into Google Cloud centralizes user login events alongside resource logs, so an unusual login in the Admin console context can be joined to subsequent resource access in Cloud Logging. Identity and Access Management (IAM) (opens in a new tab) governance depends on joining that login signal to the later privilege use.

2.4Querying and Correlating Logs Across Services and Timeframes

Correlation across services and timeframes is performed in Logs Explorer, the Cloud Logging API, and the command-line interface, with long-term analysis moved to exported storage. An investigator can view logs in the Google Cloud console under Logging and Logs Explorer for a selected project, or query entries programmatically with entries.list for repeatable forensic searches. All audit logs have a retention period, so enterprises that need longer history must create a log sink that routes entries to BigQuery or other sink destinations for storage over time and aggregate analysis.

A focused forensic query for IAM lifecycle events follows an ordered workflow:

1. Select scope: Open Logs Explorer and select the Google Cloud project that owns the affected resources.
2. Filter log and resource: Filter by audit log name and by resource.type and resource.labels.project_id to limit results to the project and service account resources.
3. Filter methods: Filter protoPayload.methodName to SetIamPolicy, google.iam.admin.v1.CreateServiceAccount, and google.iam.admin.v1.CreateServiceAccountKey to surface policy changes, new accounts, and new keys.
4. Run and expand: Run the query, then expand the time range and add Data Access and Policy Denied filters to link the administrative change to subsequent data reads, writes, and denials.

Exporting to BigQuery lets the same investigation view a subset of columns and selected data over time or other dimensions, which is necessary when an incident spans projects, folders, or an organization. Google Cloud projects contain only audit logs for resources directly within the project, while folders, organizations, and billing accounts contain logs for the entity itself, so organization-level correlation requires querying at the correct hierarchy level and retaining enough history to cover the full attack timeline.

2.5Turning Log Signals into Alerts and Preserved Evidence

Forensic capability depends on preserving logs and converting repeated patterns into alerts. Organizations set up policies and procedures to monitor administrator activities and create log-based alerts in Cloud Logging to flag suspicious API requests such as unauthorized VM creations or IAM privilege escalations. Because Admin Activity logs are always written even if the Cloud Logging API is disabled, alerting on them provides a durable baseline, while explicitly enabled Data Access logs provide the data-level visibility needed to confirm whether accessed data was actually read or modified.

A log sink in Cloud Logging (opens in a new tab) is a routing rule that sends copies of log entries that match a filter to a supported destination for retention, analysis, or forwarding. Log entries flow from the originating project, folder, or organization to the sink, and the sink then writes matching entries to its configured destination. Routing does not remove the original entries from Cloud Logging, so retention, encryption, and cost controls still apply in both places.

3.1Why Route Audit Logs

Routing audit logs gives longer preservation and stronger analysis than keeping logs only in their originating log buckets. Organizations route copies of audit logs to meet compliance requirements for creation and preservation, to keep logs beyond the bucket retention period, and to enable powerful search and centralized security analysis. A sink can route some or all logs, so critical security events can be preserved centrally while noisy, low-value volume is filtered out.

Routing also solves scale across many projects. An organization with dozens or hundreds of projects cannot practically check each project bucket, so it uses centralized routing to collect audit logs in one place. This central copy then supports investigation, compliance reporting, and forwarding to analytics platforms and third parties.

3.2Supported Destinations

Cloud Logging sinks route audit logs to Cloud Storage, BigQuery, Pub/Sub, and Cloud Logging buckets. Each destination receives data from the sink and serves a different operational need, from low-cost preservation to real-time forwarding and interactive analysis.

Traffic flows from the sink to only its configured destination, and downstream systems depend on that destination. A third-party SIEM pulls logs after the sink aggregates them in Pub/Sub or Cloud Storage, while Google Security Operations can use built-in direct ingestion for a predefined subset of logs or receive an expanded set through a sink. If a central Logging bucket or central BigQuery dataset already exists, reuse that single destination and update the existing aggregated sink with the audit log filter instead of creating parallel destinations.

3.3Aggregated Sinks for Organization-Wide Coverage

An aggregated sink is a sink created at the organization or folder level that routes logs from any or all child Google Cloud projects to one destination. Log entries flow from each child project up to the aggregated sink, and the sink then writes them to the central Cloud Storage bucket, BigQuery dataset, Pub/Sub topic, or Logging bucket. This structure lets a central security team own one destination and one set of access controls while still capturing activity from the entire resource hierarchy.

Deploying an aggregated sink follows an ordered dependency chain where each step depends on the previous one:

1. Set up a single destination: Create or select the central Cloud Storage bucket, BigQuery dataset, Pub/Sub topic, or Logging bucket in the aggregation project.
2. Create the aggregated log sink: Create the sink at the organization or folder with a filter that selects the required audit logs.
3. Grant write access to the sink: Grant the sink's writer identity permission to write to the destination, because the destination rejects writes until this trust is established.
4. Configure read access to the destination: Grant analysts, auditors, and services permission to read the central destination without granting them access to every source project.
5. Verify routed logs: Confirm that audit entries from multiple projects arrive in the destination and match the intended filter.

3.4Filters and Exclusions to Control Volume and Cost

Sink filters and exclusions control which audit logs are routed and which are discarded to manage log volume and cost while preserving critical security events. The filter on a sink selects matching entries for delivery, while an exclusion prevents matching entries from being stored in Cloud Logging or routed onward. Because Data Access audit logs record read operations and can be very large, they are the primary driver of volume and storage cost.

Admin Activity audit logs are always written and cannot be configured, excluded, or disabled, even if the Cloud Logging API is disabled. Data Access audit logs are optional and high-volume, so exclusions are normally applied to Data Access logs rather than to Admin Activity logs. A common tradeoff is to retain all Data Access logs for authentication and authorization services and for production storage systems, while excluding Data Access logs in development projects where the volume has little security value.

3.5Retention, Encryption, and Cost Controls

Retention for audit logs kept in Cloud Logging is governed by the log bucket where the logs are held, not by the log type. Custom retention can be configured between 1 day and 3650 days to meet compliance needs, and a custom rule applies to all logs in the bucket regardless of log type or whether the log was copied from another location. Logs routed to Cloud Storage, BigQuery, or Pub/Sub (opens in a new tab) are subject to those services' own storage and pricing in addition to Cloud Logging retention.

By default, Cloud Logging encrypts customer content stored at rest with Google-managed keys. When an organization has advanced encryption requirements, it configures customer-managed encryption keys (CMEK) to control and manage the key encryption keys that protect log storage. The log bucket depends on the CMEK configuration, so loss of access to the key affects access to the stored audit logs.

Cost control depends on estimating usage and excluding low-value logs before they consume allotments and storage. Data Access audit logs can push projects over log allotments and incur additional storage charges in Cloud Logging and in routed destinations. Effective practice is to view usage data, configure alerting on usage and billing, and apply exclusions to audit logs that are not useful for security or compliance.

Key Facts

• Admin Activity audit logs are always written and cannot be configured, excluded, or disabled, even if the Cloud Logging API is disabled.
• Data Access audit logs must be explicitly enabled in the IAM AuditConfig object and cover ADMIN_READ, DATA_READ, and DATA_WRITE operations.
• Log entries flow from the originating project to a sink, and the sink writes matching entries to Cloud Storage, BigQuery, Pub/Sub, or a central Cloud Logging bucket.
• Custom log bucket retention can be configured from 1 day to 3650 days, with Bucket Lock enforcing retention so audit evidence cannot be deleted early.
• Reading Data Access logs in the _Default bucket requires the Private Logs Viewer role, while the Logs Viewer role can read only Admin Activity, Policy Denied, and System Event logs.