Prepare and test your skills
Prepare and test your skills
Log redaction with Cloud Logging is the practice of preventing, hiding, or transforming regulated fields such as personally identifiable information (PII) and protected health information (PHI) while keeping logs useful for debugging and auditing. Redaction starts before ingestion by keeping sensitive values out of payloads and metadata, continues at read time with views and field masks, and extends to transformation when utility must be preserved. Audit logs never include resource data contents or query results, but they can capture resource metadata, so names, labels, and request fields must be treated as a redaction boundary. When transformed data must be analyzed, it flows to BigQuery (opens in a new tab) for analysis of data profiles.
Prevention is the most reliable redaction control because data that is never logged cannot leak through viewers, exports, or analytics. Producing workloads must not place PII, PHI, or credentials in experiment output, secondary log servers, resource metadata, build files, or request fields that Cloud Logging will capture. This rule applies across services where operational fields flow directly from the source system to log storage, so the producing workload depends on developer discipline to keep sensitive values out.
Sensitive values commonly leak through operational fields that appear harmless during development:
IntegrationParameter values, Integration Connectors connection names and Connection configuration settings, and Knowledge Catalog lineage Process.attributes and Run.attributes valuesWhen payloads must contain sensitive data, configure access control for the logs that will store the payload and write any Sensitive Data Protection output only to storage targets inside the secure environment.
Visibility controls in Cloud Logging separate what is stored in a log bucket from what each principal can read at query time. A log view hides entire LogEntry objects that do not match its filter, while field-level access controls hide individual fields inside the LogEntry, such as an email address in the payload. The log bucket stores the data, the view or field mask applies at read time, and Identity and Access Management (IAM) bindings decide which principal sees which view.
| Mechanism | Scope of hiding | Choose when |
|---|---|---|
| Log views on a log bucket | Entire LogEntry that does not match the view filter | Different teams should see different subsets of logs, such as only production errors versus all platform logs |
| Field-level access controls on a log bucket | Individual LogEntry fields, such as a PII field in the payload | Most users need the log for debugging but must not see one regulated field inside it |
Field-level access is the finer redaction tool, but it carries an analytics tradeoff. Do not configure field-level access controls on a bucket that must be analyzed with Observability Analytics, because Observability Analytics cannot query log buckets that have field-level access controls configured. Choose log views when analytics compatibility matters, and choose field-level masking only when field-granular hiding outweighs that loss.
Exam tip: Log views redact whole entries while field-level access redacts individual fields, but only log views remain compatible with Observability Analytics.
Transformation with Sensitive Data Protection (formerly Cloud DLP) redacts sensitive values while preserving log and dataset utility for debugging, measurement, and auditing. Sensitive Data Protection discovers, classifies, and de-identifies PII and PHI through pseudonymization, de-identification, tokenization, and generalization such as bucketing, so an analyst can join or count records without seeing raw identifiers. Data flows from the source log or table through an inspection job to a de-identified output, and that output then flows to the governed destination for analysis.
Apply transformation before persistence when raw values are not needed downstream, and encrypt individual values that must remain reversible. Use GoogleSQL authenticated encryption with associated data (AEAD) functions to encrypt single values inside a table, use customer-managed encryption keys (CMEK) in Cloud Key Management Service (Cloud KMS) to control table-level encryption, and use tokenization pipelines that ingest sensitive data, replace it with tokens, and persist only tokens to BigQuery. Label datasets from Sensitive Data Protection findings so training and analytics jobs can route around data still marked sensitive, and enable Model Armor sensitive-data filters to discover and block PII that would otherwise be exposed through model prompts and outputs.
Secure destinations keep redacted logs available to the right readers for the required retention period without recreating exposure. Export audit logs from Cloud Logging to Cloud Storage for long-term archival and to BigQuery for analytical, monitoring, and forensic use, then configure access controls on those destinations separately from the source project. Readers with the Logs Viewer role (roles/logging.viewer) can access Admin Activity audit logs, while readers need the Private Logs Viewer role (roles/logging.privateLogViewer) for Data Access audit logs, so the export destination inherits a trust boundary outside the control of the team that produced the log.
Retain and review logs as a closed loop that supports both debugging and compliance. Configure custom retention on the log bucket between 1 day and 3650 days so all logs in the bucket share one enforceable lifetime, protect buckets holding audit logs with CMEK when default Google-managed encryption does not meet requirements, and exclude audit logs that have no debugging or compliance value to reduce cost and exposure. Regularly review audit logs in BigQuery or a security information and event management (SIEM) platform, validate collection in a test project before enabling organization-wide Data Access audit logs, and keep Data Access logs enabled where possible because they help support troubleshoot access without granting broader data access.
Identifying and classifying sensitive data in log streams is the process of finding where regulated information appears in operational telemetry and labeling it by type and risk before it is stored or exported. Cloud Logging collects text payloads, resource metadata, labels, and audit events from across Google Cloud, and any of those fields can carry PII or PHI if applications or configurations place it there. Classification uses Sensitive Data Protection infoTypes, sensitivity levels, and data risk levels to distinguish direct identifiers, quasi-identifiers, and other regulated categories. Correct identification determines whether a log stream needs redaction, tighter access, or exclusion from training and analytics.
Regulated data categories in logs group sensitive values by how directly they identify a person and what compliance rule applies to them. Sensitive Data Protection provides the classification schema through infoTypes, which are named detectors for patterns such as credit card numbers, addresses, phone numbers, and email addresses, plus sensitivity levels and data risk levels that summarize exposure. Data minimization requires that logs do not collect, keep, or use data that is not strictly necessary, so classification starts by deciding whether PII or PHI should be in the log stream at all.
| Category | What it identifies | Typical log pattern |
|---|---|---|
| Direct identifiers | A single value that identifies a person on its own | Credit card number, phone number, email address, or patient name in a message payload or URI |
| Quasi-identifiers | Values that identify a person only when combined | ZIP code plus birth date, device type plus timestamp, or department plus rare diagnosis across correlated log fields |
| Other regulated data | Credentials and health or financial context governed by HIPAA (opens in a new tab) and similar rules | Security credentials, diagnosis codes, treatment notes, or financial account details in metadata, labels, or integration parameters |
Quasi-identifiers are the most missed risk in log streams because each field looks harmless alone but becomes identifying when Cloud Logging, BigQuery, or a SIEM joins logs by time, user, or resource. That is why discovery reports break findings down by infoType, by location, and by risk, rather than by single-field matches alone.
High-risk log sources and content patterns are the places where PII and PHI leak into logs even when the main application data is protected. Audit logs never include resource data contents or query results, but they do capture resource metadata, so PHI or credentials placed in names, labels, or configuration will flow from the source service to Cloud Logging and then to any export destination. Application code that logs request bodies, error details, or experiment output creates the same flow from the workload to log buckets, BigQuery datasets, and Cloud Storage archives.
Common patterns that place sensitive data into log streams include:
IntegrationParameter values, Integration Connectors connection names and configuration, and Cloud CDN cached responses containing PHIProcess.attributes and Run.attributes fieldsExam tip: Audit logs capture resource metadata but never capture resource contents or query results, so never place PHI, PII, or credentials in a name, label, header, or lineage attribute that logs will retain.
Classification schemas and discovery with Sensitive Data Protection turn scattered log findings into a consistent inventory with sensitivity and risk. Discovery runs continuously at the organization, folder, or project level and generates data profiles containing metrics and insights about sensitivity levels and data risk levels for BigQuery, Cloud Storage, and Vertex AI datasets. Data flows from scanned tables and buckets to data profiles, and profiles can then flow to Security Command Center, Google SecOps, and Knowledge Catalog to enrich security posture, threat investigation, and governance with business metadata and lineage.
A practical identification workflow for log-related data follows an ordered sequence:
When discovery finds sensitive data that should not be broadly accessible, the same classification guides remediation without losing utility. Teams create a de-identified copy that masks or tokenizes sensitive columns using pseudonymization, de-identification, and generalization such as bucketing, so analysts can work in BigQuery or Cloud Storage without seeing raw identifiers. Access to the original log exports and profile outputs is then limited with IAM roles, where the Logs Viewer role grants Admin Activity audit logs and the Private Logs Viewer role grants Data Access audit logs, and exports are sent to Cloud Storage for long-term archival and to BigQuery for analysis with appropriate access controls on each destination.
Compliance and security implications of log data handling are the legal, residency, retention, and access risks created when logs capture regulated data such as PII and PHI. In Google Cloud, Cloud Logging stores, indexes, and searches log data in regional buckets, and that storage choice, retention rule, encryption control, and access grant together determine whether handling meets frameworks such as GDPR and HIPAA. A compliant strategy therefore minimizes what is logged, regionalizes where it is stored, limits how long it is kept, encrypts it with the required key control, and restricts who can read it.
Data residency for logs means the log bucket infrastructure that stores, indexes, and searches logs lives in a specific geographic location chosen to meet latency, availability, and compliance needs. Cloud Logging buckets are regional resources, so an organization required to keep logs data in specific regions configures a default resource location that automatically applies to new _Default and _Required buckets and creates user-defined buckets to centralize or subdivide storage by region. The same residency principle extends to log destinations and related services: Cloud Storage stores or replicates data only within the specified region, Pub/Sub restricts message storage with message storage policies, and Assured Workloads establishes data boundaries for sensitive and regulated workloads. Traffic flows from Google Cloud services to the regional bucket and then, through log sinks, to export destinations, so the perimeter decision controls both where PII or PHI rests and which jurisdictions govern it.
Retention policy for logs is the bucket-level rule that defines how long every log in that bucket is kept before deletion, and it directly affects GDPR minimization and HIPAA audit-trail duties. Cloud Logging allows custom retention between 1 day and 3650 days, and the custom rule applies to all logs in a bucket regardless of log type or whether the log was copied from another location. By default, Admin Activity and System Event audit logs are retained for 400 days while Data Access audit logs are retained for 30 days, after which the organization must move audit logs to another Cloud Storage or database product that meets its retention need. Organizations apply data minimization alongside retention by not collecting, keeping, or using data that is not strictly necessary, by using synthetic or fully anonymized data where possible, and by using Cloud Storage lifecycle controls and Object Versioning with retention policy locks to preserve archives and recover from accidental deletion.
Access control for sensitive log data is the layered system that decides which identity can see which log entries or fields inside a project, bucket, or export. IAM provides the base layer, where users with the Logs Viewer role can access Admin Activity audit logs and users with the Private Logs Viewer role can access Data Access audit logs, with separation of duties, least-privilege service accounts, IAM Conditions, principal access boundary (PAB) policies, and just-in-time elevation through Privileged Access Manager further limiting privilege. Two bucket-level mechanisms add granularity for PII and PHI, while centralized sinks isolate audit logs outside the workload team's control:
LogEntry from users who should not see that subset of logs stored in the same bucket.LogEntry fields, such as an email address in the payload, from most users while leaving the rest of the entry visible.| Control | What it hides | Choose when |
|---|---|---|
| IAM log roles | Buckets or log types by role grant | The requirement is who can read Admin Activity versus Data Access audit logs |
| Log views | Entire LogEntry objects in a subset | Different teams sharing one bucket need different full-entry visibility |
| Field-level access controls | Individual fields inside LogEntry | Most users need the log but must not see PII or PHI fields in the payload |
Exam tip: Log views hide the whole entry while field-level access controls hide only selected fields, and Observability Analytics cannot be used on a log bucket that has field-level access controls configured.
Protective controls for logs combine encryption at rest and in transit, network perimeters, and de-identification so that sensitive fields are unreadable or never stored. Cloud Logging encrypts customer content at rest by default, and organizations with advanced requirements configure customer-managed encryption keys (CMEKs) in Cloud Key Management Service (Cloud KMS) to control their own key encryption keys for log buckets. Data flows across trust boundaries under VPC Service Controls, which places sensitive data in its own service perimeter around Google-managed services and uses ingress and egress rules plus firewall rules that deny all traffic by default to prevent exfiltration. For utility without exposure, Sensitive Data Protection discovers, classifies, and then de-identifies data through pseudonymization, tokenization, masking, and generalization such as bucketing before analysts work with it.
Regulatory alignment for log handling means processing choices for residency, retention, and access map to specific HIPAA (opens in a new tab) and data-protection obligations and remain auditable. Under HIPAA practices, the organization enters into a Google Cloud Business Associate Agreement (BAA), disables unsupported services when working with PHI, does not use pre-GA offerings with PHI, and avoids including PHI or security credentials in resource metadata because audit logs never include resource data contents or query results but might capture resource metadata. Related prohibitions reduce accidental PHI spread through observability and integration fields.
The related prohibitions that reduce accidental PHI spread are:
Accountability depends on lineage and audit trails that show who accessed or modified data and when. Knowledge Catalog centrally discovers, manages, and governs data and AI artifacts with data lineage, quality checks, and business glossary, while Data Access audit logs, Admin Activity audit logs, and Privileged Access Manager audit logs record API calls and administrative actions. Data profiles from Sensitive Data Protection discovery can be exported to Security Command Center (opens in a new tab), Google SecOps, and Knowledge Catalog, and audit logs can be exported to Cloud Storage for long-term archival and to BigQuery for analytical, monitoring, and forensic needs with appropriate access controls on those destinations.
Log views hide entire LogEntry objects that do not match the view filter, while field-level access controls hide individual fields inside a LogEntry such as an email address in the payload. Field-level access controls should not be configured on a bucket that must be analyzed with Observability Analytics because Observability Analytics cannot query log buckets that have field-level access controls configured.
Admin Activity and System Event audit logs are retained for 400 days by default while Data Access audit logs are retained for 30 days. Custom retention between 1 day and 3650 days can be configured on the log bucket and applies to all logs in that bucket regardless of log type.
The Logs Viewer role grants access to Admin Activity audit logs while the Private Logs Viewer role is required for Data Access audit logs. Export destinations in Cloud Storage and BigQuery require access controls configured separately from the source project.
An organization is running containerized microservices where application logs occasionally contain sensitive user data, such as jsonPayload.user_email and jsonPayload.tax_id.
DevOps engineers require continuous access to these logs in Logs Explorer to debug production incidents, inspect error codes, and verify request latency. However, security compliance requires that standard engineers cannot view the sensitive PII fields, while authorized compliance auditors must retain the ability to inspect the full, unredacted log entries.
Which native Cloud Logging strategy should you implement to meet these requirements?
Enable Log Analytics on the bucket, configure field-level access controls, and use SQL views to mask the sensitive JSON payload keys.
Configure field-level access controls on the log bucket by setting the restricted fields list, and grant the roles/logging.fieldAccessor IAM role exclusively to the compliance auditors.
Define an exclusion filter on the _Default log sink matching jsonPayload.user_email:* OR jsonPayload.tax_id:* so that sensitive fields are stripped before ingestion.
Create two separate Log Views on the log bucket, applying a filter on the engineers' Log View to hide log entries containing jsonPayload.user_email and jsonPayload.tax_id.
Last updated