Prepare and test your skills
Prepare and test your skills
Log export and retention compliance is the design that keeps audit logs searchable, stored in the right location, and available long enough for forensics and audits. Cloud Logging (opens in a new tab) first stores logs in log buckets in the project where they were generated. Log sinks (opens in a new tab) then route copies of those logs to long-term or analytic destinations outside that short-term storage, while retention rules on the buckets control how long the local copy is kept.
A log sink is a routing rule in Cloud Logging that sends some or all logs that match its filter to a destination outside the source bucket. Sinks are used when an organization must keep audit logs longer than bucket retention allows, needs more powerful search, or must forward logs to security tools and third parties. A sink routes a copy, so the original handling in the source bucket still applies unless it is separately changed. The three main destinations differ in cost, analysis power, and delivery speed:
Cloud Logging's Log Router stores audit logs in the _Required and _Default buckets, while log sinks route copies to Cloud Storage for long-term archiving, BigQuery for SQL analysis, or Pub/Sub for real-time delivery to a SIEM.
| Destination | What it provides | Choose when |
|---|---|---|
| Cloud Storage | Low-cost object storage for log copies | Long-term archiving is needed for regulatory retention and forensic re-analysis |
| BigQuery | SQL-based dataset for powerful search and analysis | Large volumes of audit logs must be queried quickly across time, projects, and services |
| Pub/Sub | Real-time message delivery to applications and other stores | Logs must flow from Pub/Sub to a SIEM, automation, or an external compliance store |
Traffic therefore flows from the source log bucket through the sink filter to Cloud Storage for archiving, to BigQuery for analysis, or to Pub/Sub for real-time forwarding.
An aggregated sink is a sink created at the organization or folder level that routes logs from any or all Google Cloud projects under it. Aggregated sinks give central security and compliance teams one place to collect Admin Activity, Data Access, System Event, and Policy Denied audit logs without configuring each project separately. Log entries flow from the source projects to the central destination named in the aggregated sink. This central flow supports consistent retention and forensic analysis across the organization.
Log buckets are the containers in Cloud Logging that store and organize log data. For every billing account, project, folder, and organization, Cloud Logging automatically creates two buckets, _Required and _Default, with correspondingly named sinks. The _Required bucket stores Admin Activity audit logs and System Event audit logs, and neither its storage nor its sink can be disabled or reconfigured. Admin Activity audit logs are always written, cannot be configured, excluded, or disabled, and are still generated even if the Cloud Logging API is disabled.
Admin Activity and System Event logs are always stored in the _Required bucket in the project where they were generated. If those logs are routed to a different project, they do not pass through the destination project's _Default or _Required sink and are not stored there, so a separate sink must be created in the destination project to store them. The _Default bucket stores enabled Data Access audit logs and Policy Denied audit logs by default. Data Access storage is stopped by disabling those logs, while Policy Denied storage is stopped by excluding them in the sink filter.
Exam tip: A routed copy of an Admin Activity log does not automatically land in the destination project's buckets — create a sink in the destination project if that copy must also be stored there.
A retention period is the custom rule on a log bucket that defines how long Cloud Logging keeps every log in that bucket. To meet compliance needs, retention can be configured between 1 day and 3650 days, which allows up to about ten years of local retention for long-term investigations. A custom retention rule applies to all logs in the bucket regardless of log type or whether the log was copied from another location. Separate buckets are therefore needed when different data types require different retention periods.
Data residency for logs is controlled by where buckets and sink destinations are located, including default resource locations configured for the organization. Choosing bucket regions and Cloud Storage or BigQuery dataset locations in approved regions keeps forensic copies inside required geographic boundaries while still allowing centralized analysis. By default, Cloud Logging encrypts stored content at rest, and organizations with stricter requirements can use customer-managed encryption keys (CMEK) to control the key encryption keys that protect log storage. Long-term retention increases volume and cost, especially because Data Access audit logs can be very large. Costs are managed by estimating usage, setting alerting on usage, and creating sink exclusions for audit logs that are not useful, such as excluding Data Access logs from development projects while keeping them for production and sensitive data access.
Security analysis and incident response through logging is the practice of collecting audit and network logs in one place, searching them for signs of unauthorized access, and turning suspicious patterns into alerts for fast response. In Google Cloud, Cloud Logging stores the logs, Logs Explorer and Log Analytics provide search and SQL analysis, Cloud Monitoring sends alerts, and Security Command Center correlates findings with log evidence. Together they give visibility into administrative actions and data access across projects, folders, and organizations.
Cloud Audit Logs are the core record of who performed an action, what action was performed, where it happened, and when it happened. Each entry stores an AuditLog object in the protoPayload field, including user identity in AuthenticationInfo, event type in methodName, timestamp, success or failure status, request details, and affected service or resource. Audit entries also capture origination and the identity of the affected data or system component:
Visibility into Google personnel is covered by Access Transparency and Access Approval. Cloud Audit Logs show actions by your own admins, while Access Transparency provides near real-time logs when Google Cloud admins access your content. Access Approval lets you explicitly approve access to data or configurations before Google Support or Engineering access occurs and closes the gap where a support ticket requiring data access would not have been tracked.
Exam tip: Admin Activity logs are always enabled, while Data Access logs are disabled by default except for BigQuery — enable Data Access logs only where data-access visibility justifies the volume and cost.
Investigation in Logs Explorer is the process of searching audit logs for activity by a compromised identity and for new resources created during the incident window. Access to logs depends on Identity and Access Management roles granted at the project, folder, or organization level. The Logs Viewer role (roles/logging.viewer) allows viewing audit logs and listing log entries, the Private Logs Viewer role (roles/logging.privateLogViewer) adds permission to view Data Access audit logs with private entries, and the Security Center Findings Viewer role (roles/securitycenter.findingsViewer) allows listing and getting findings in Security Command Center. An investigation follows an ordered flow from logs to findings to containment:
protoPayload.authenticationInfo.serviceAccountDelegationInfo matches the compromised principal.access.principal_email="USER_EMAIL" to display all findings caused by the same user.A security log analytics pipeline (opens in a new tab) is an organization-wide workflow that moves logs from every project, folder, and billing account into a single analytics destination for SQL or detection-rule analysis. Data flows from Google Cloud services to Cloud Logging, then through an aggregated sink to the chosen destination, then into queries that answer login, permission-change, provisioning, workload, data-access, and network-security questions. The pipeline is built in three ordered stages:
A log scoping tool helps choose which logs to enable by mapping each log type to MITRE ATT&CK tactics and techniques, CIS Google Cloud Platform compliance checks, and Event Threat Detection rules that depend on that log. The tool generates a log filter as logs are selected, and that filter is reused later when configuring the aggregated sink.
Network and intrusion logs record traffic and attack attempts so anomalous infrastructure behavior can be detected, audited, and investigated in near real time. Traffic flows from VM instances and firewall enforcement points to Cloud Logging, where the entries support monitoring, forensics, and real-time security analysis. VPC Flow Logs record network traffic flows sent from or received by virtual machine instances. Each VPC subnet can have Flow Logs enabled or disabled independently, so volume can be reduced by enabling them only on the in-scope cardholder data environment. Flow Logs combined with egress firewall rules limit outbound traffic to authorized endpoints in a way that is auditable and difficult to circumvent, a pattern used for PCI Data Security Standard compliance (opens in a new tab).
Firewall Rules Logging provides per-rule auditing of TCP and UDP connections inside a VPC for rules you create. Logging is enabled individually for each firewall rule whose connections need to be recorded. The entries verify whether a deny rule is functioning as intended and provide early warning that the network is being used in an unapproved manner. For detail beyond flows, such as individual HTTP requests, outbound requests can be proxied through redundant reverse proxy servers that forward access logs to Logging. Cloud Intrusion Detection System (Cloud IDS) provides threat detection for intrusions, malware, spyware, and command-and-control attacks on the network. Cloud IDS consumes mirrored traffic through Packet Mirroring to gain full visibility into north-south and east-west traffic, including VM-to-VM communication that reveals lateral movement.
Metrics, alerts, and retention turn stored logs into continuous monitoring and long-term evidence. Log data flows from Logging to Monitoring for alerting and to Cloud Storage or BigQuery for retention and large-scale analysis. Monitoring is configured to send alerts when something goes wrong, covering environmental, audit, and internal application events. The alerting strategy is based on risk and attack vectors for each component, such as triggering alerts when intrusion detection reports successful or unsuccessful intrusion attempts or when Firewall Rules Logging shows attempts to violate specific network policies. For large datasets, logs are routed through sinks to BigQuery, which is optimized to query large log volumes, and direct logging to BigQuery is used for logs that require near real-time analysis. For compliance, every server activity and user action must remain examinable later, so entries are exported to Cloud Storage, BigQuery, or Pub/Sub for longer retention and aggregated exports are used to centralize review across the organization.
Cloud Audit Logs (opens in a new tab) is the Google Cloud service that records who did what, where, and when within Google Cloud resources. Logs are written for each Google Cloud project, folder, organization, and billing account, and projects contain only the audit logs for resources directly within that project. The service provides four categories: Admin Activity, Data Access, System Event, and Policy Denied logs. Together these categories give visibility into administrative actions and data access needed for security and compliance.
The four audit log categories divide audited operations by the kind of action performed. Admin Activity logs record admin write operations that write metadata or configuration information, such as creating or updating clusters, changing Identity and Access Management (IAM) policy, or initializing configuration. Data Access logs record admin read operations that read metadata or configuration plus data read and data write operations that read or write user-provided data, such as getting or listing clusters, reading reports, or streaming resources. System Event and Policy Denied logs complete the set and are stored as their own log types:
| Category | Log name suffix | Operations documented |
|---|---|---|
| Admin Activity audit logs | cloudaudit.googleapis.com%2Factivity | Admin write operations that write metadata or configuration |
| Data Access audit logs | cloudaudit.googleapis.com%2Fdata_access | Admin read, data read, and data write operations |
| System Event audit logs | cloudaudit.googleapis.com%2Fsystem_event | System Event audit logging data |
| Policy Denied audit logs | cloudaudit.googleapis.com%2Fpolicy | Policy Denied audit logging data |
Admin Activity logs track resource modifications, while Data Access logs track sensitive data interactions but are much larger in volume. Data Access logs are disabled by default and are not written unless explicitly enabled, because they can be quite large, with the exception of Data Access logs for BigQuery which cannot be disabled. Admin Activity logs are always enabled and cannot be disabled, so a baseline of configuration changes is always retained.
Exam tip: Admin Activity logs cannot be disabled, while Data Access logs must be explicitly enabled before they are written.
An audit log entry is a Cloud Logging LogEntry object whose protoPayload field contains an AuditLog object. The AuditLog object holds the audit data, including serviceName that identifies the Google Cloud product that wrote the entry, such as compute.googleapis.com or gkemulticloud.googleapis.com, plus authentication information and caller IP in RequestMetadata.caller_ip. For managed Kubernetes, Google Kubernetes Engine (GKE) (opens in a new tab) also writes entries that follow this same LogEntry and AuditLog structure. Optional service-specific detail appears in the metadata field for newer integrations or the serviceData field for earlier integrations:
projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity, abbreviated as activity, data_access, system_event, and policy.AuditLog data, including serviceName, authentication info, and service-specific metadata or serviceData.The part of the log name after /logs/ is URL-encoded, so / appears as %2F. Because every entry names its owner, target resource, time, and writing service, logs can be queried in Logs Explorer by logName, resource.type, and protoPayload.serviceName to trace an action from caller to target.
Configuration of Cloud Audit Logs controls which logs flow to storage and who can read them. Data Access logs must be turned on per project, folder, or organization when read visibility is required, while Admin Activity logs flow automatically. Once written, logs flow to the Log Router in Cloud Logging, where the router ingests logs into Cloud Logging storage, excludes logs with exclusion filters, or exports logs to BigQuery, Cloud Storage, and Pub/Sub for custom aggregation and long-term retention. Cost and access control depend on filtering and bucket placement. Exclusion filters reduce the volume sent to Cloud Logging, which is important because Data Access and control-plane logs can consume significant ingestion quota and storage cost. Storing sensitive logs in a separate log bucket with limited access prevents users with broad viewer access from seeing them and allows deletion for privacy without affecting other logs. The Logs Viewer role (roles/logging.viewer) grants read-only access to Admin Activity, Policy Denied, and System Event logs, while the Private Logs Viewer role (roles/logging.privateLogViewer) adds the ability to read Data Access logs in the _Default bucket.
_Default or _Required bucket unless a separate sink is created in that destination project.Admin Activity audit logs record admin write operations that write metadata or configuration and are always enabled and cannot be disabled, while Data Access audit logs record admin read, data read, and data write operations and are disabled by default except for BigQuery and must be explicitly enabled. Data Access audit logs track sensitive data interactions but are much larger in volume than Admin Activity audit logs.
Route log copies to Cloud Storage for low-cost long-term archiving for regulatory retention and forensic re-analysis, to BigQuery for SQL-based search and analysis of large volumes across time, projects, and services, and to Pub/Sub for real-time delivery to a SIEM, automation, or an external compliance store. A log sink routes a copy of matching logs, so the original handling in the source bucket still applies unless it is separately changed.
An aggregated sink is a sink created at the organization or folder level that routes logs from any or all Google Cloud projects under it to a central destination. The aggregated sink gives central security and compliance teams one place to collect Admin Activity, Data Access, System Event, and Policy Denied audit logs without configuring each project separately and supports consistent retention and forensic analysis across the organization.
Log bucket retention in Cloud Logging is a custom rule that defines how long every log in the bucket is kept and can be configured from 1 day to 3650 days for up to about ten years of local retention. The rule applies to all logs in the bucket regardless of log type or whether the log was copied from another location, so separate buckets are required when different data types need different retention periods.
An enterprise is designing a compliance and auditing architecture across its Google Cloud organization while optimizing Cloud Logging storage and ingestion costs. The security operations team has established the following requirements:
_Default log bucket.Which combination of audit log configurations and IAM roles should you implement?
Rely on the default configuration for Admin Activity and System Event logs, explicitly enable Data Access audit logs for the relevant services, and grant auditors the Private Logs Viewer (roles/logging.privateLogViewer) role.
Rely on default configuration for Data Access logs, explicitly enable Policy Denied logs, and grant auditors the Private Logs Viewer (roles/logging.privateLogViewer) role.
Enable Data Access audit logs across all services by default, create exclusion filters on Admin Activity logs to reduce costs, and grant auditors the Logs Viewer (roles/logging.viewer) role.
Explicitly enable Admin Activity and System Event audit logs in the audit configuration, disable Data Access logs, and grant auditors the Logs Viewer (roles/logging.viewer) role.
Last updated