Professional Cloud Security Engineer
Log retention is a critical component of any comprehensive logging strategy in Google Cloud. Cloud Logging retains logs according to retention rules that apply to the log bucket type where the logs are held. Organizations can configure Cloud Logging to retain logs between 1 day and 3650 days (approximately 10 years) to meet various compliance needs. Custom retention rules apply to all the logs in a bucket, regardless of the log type or whether that log has been copied from another location. Admin Activity and System Event audit logs are retained for 400 days by default, while Data Access audit logs are retained for 30 days by default. Organizations should move audit logs to another storage location at the end of this period to meet longer-term retention requirements.
Log archival involves choosing where logs are stored and implementing cost-effective storage solutions. In Logging, buckets are regional resources—the infrastructure that stores, indexes, and searches logs is located in a specific geographical location. Organizations might be required to store logs data in specific regions to meet latency, availability, or compliance requirements. To automatically apply a particular storage region to new _Default and _Required buckets created in the organization, you can configure a default resource location. For cost-effective archival, organizations can implement lifecycle management to move logs to storage classes like Coldline, which offers lower storage costs for data that is accessed less frequently.
Access controls for logs in Google Cloud involve multiple layers of protection. Field-level access controls let you hide individual LogEntry fields from users of a Google Cloud project, providing a more granular way to control the log data a user can access compared to log views, which hide the entire LogEntry. For example, you might want to redact external user PII, such as an email address contained in the log entry payload, from the majority of your organization's users. Organizations should use IAM allow policies to control access to logs and log configurations, enable uniform bucket-level access on buckets that store audit logs, and implement access controls in application code to restrict access to audit functionality.
Protecting log data involves implementing encryption and managing who can access audit information. By default, Cloud Logging encrypts customer content stored at rest. Organizations with advanced encryption requirements can configure customer-managed encryption keys (CMEK) to control and manage their own encryption instead of Google managing the key encryption keys that protect the data. To protect audit information and audit logging tools from unauthorized access, modification, and deletion, organizations should enable audit logging (including for Cloud Storage), use IAM allow policies to control access to logs and log configurations, enable uniform bucket-level access on the bucket that stores audit logs, and implement access controls in application code to restrict access to audit functionality.
Data Access audit logs help Google Support troubleshoot issues with your account, so enabling Data Access audit logs when possible is recommended. However, these logs can be large and may incur additional costs for storage. Organizations should evaluate whether to enable Data Access audit logs when enabling new Google Cloud services. After defining an organization-level data access policy and enabling Data Access audit logs, use a test Google Cloud project to validate the configuration of audit logs collection before creating developer and production projects in the organization. If you plan to use Log Analytics to analyze audit logs, don't configure field-level access controls on the log bucket that stores those logs, as Log Analytics cannot be used on log buckets that have field-level access controls configured.
An effective logging strategy is the foundation of security visibility and proactive threat detection in Google Cloud. It involves a systematic workflow: enabling relevant security logs, routing them to a centralized analytics destination, and analyzing them with purpose-built queries or rules. This process ensures you can audit cloud usage, detect malicious activity like malware or unauthorized access, and support compliance requirements. Key to this strategy is prioritizing which logs to enable based on your security needs, as some logs (like Data Access audit logs) incur ingestion costs and are not enabled by default.
The first step is identifying and enabling the logs that provide the necessary visibility. Tools like the log scoping tool map specific Google Cloud logs (e.g., Cloud Audit Logs, VPC Flow Logs, DNS logs) to threat frameworks like the MITRE ATT&CK® Matrix, helping you select logs based on the threat tactics you need to monitor. After selection, you must enable these logs at the service level, which may include activating Data Access audit logs for granular activity tracking. Once enabled, logs should be aggregated and routed from your entire organization to a centralized destination using a Cloud Logging aggregated sink. Common destinations include a log bucket for Log Analytics, a BigQuery dataset, or a Pub/Sub topic for integration with external SIEM solutions.
With logs centralized, the next phase is analysis to transform raw data into security insights. This is achieved by running detection logic against the log streams. In Google Cloud, you can leverage pre-built SQL queries from the Community Security Analytics (CSA) project for use in Log Analytics or BigQuery. These queries answer critical security questions, such as identifying failed login attempts, detecting permission changes, or spotting data exfiltration patterns. For more advanced, real-time detection, you can integrate logs with Security Command Center Premium, which uses managed detectors for Event Threat Detection, or stream them to a SIEM like Google Security Operations for correlation and advanced threat hunting using custom YARA-L rules.
To move beyond manual querying, you can automate log analysis and integrate findings into your security operations workflow. This involves creating custom detection rules and alerts within Cloud Monitoring or Security Command Center to notify teams of anomalous activities. Furthermore, you can build automated pipelines that query logs in a linked BigQuery dataset and push custom findings directly to Security Command Center. Exporting Security Command Center findings to a Pub/Sub topic allows for seamless integration with your existing SIEM or SOAR platforms, enabling centralized triage and response. This closed-loop process—from log collection to automated alerting and integration—ensures a robust, responsive threat detection capability.
Google Cloud provides several core telemetry sources to maintain comprehensive security visibility and meet regulatory compliance requirements. Admin Activity audit logs and System Event logs are automatically enabled across Google Cloud Platform services to record configuration changes and infrastructure events. In contrast, Data Access audit logs capture data read and data write operations but must be explicitly enabled for most services due to potential storage volume. Additionally, administrators should enable network telemetry like VPC Flow Logs and integrate Google Workspace audit logs to ensure continuous network traffic inspection and identity monitoring.
Centralizing log collection relies on the Log Router to process incoming log entries and distribute them to designated storage sinks based on security needs. Security teams design sinks to route critical telemetry to specific destinations based on compliance and analysis requirements:
Configuring these sinks ensures that all critical security events reach centralized monitoring tools without manual intervention. To safeguard data pipelines, proper IAM permissions must be granted to the designated sink service accounts.
Managing ingestion volume and pipeline performance requires balancing detailed security visibility with cost optimization. Administrators can apply exclusion filters to sinks within the Log Router to discard high-volume, low-priority logs such as routine Google Kubernetes Engine system metrics. Alternatively, exempting service accounts directly through IAM audit configuration policies is the recommended approach because it prevents unnecessary log creation at the API source. Applying these controls allows organizations to suppress duplicate or high-frequency records while preserving vital audit trails for incident response.
Finally, organizations must configure data governance and access boundaries for their ingested logs. Custom retention policies can be applied to regional log buckets to retain records for periods ranging between 1 day and 3,650 days. To meet advanced compliance mandates, storage buckets can be protected using customer-managed encryption keys (CMEK) to enforce independent key management. Furthermore, administrators can configure field-level access controls to mask sensitive user data, while using IAM allow policies to restrict access and preserve the integrity of all collected logs.
Gauge your current knowledge
Gauge your current knowledge
StudyCerts
StudyCerts