Professional Cloud Security Engineer
Prepare and test your skills
Prepare and test your skills
Sensitive Data Protection (SDP) offers two key services. The first is discovery, a continuous scanning service that builds data profiles of your information across Cloud Storage, BigQuery, and Datastore. These profiles give you a high-level view of where sensitive data resides and its risk level across your entire organization, folder, or project. You can automate this by setting up organization-level discovery profiles, which automatically scan and classify new data as it is created, providing a central source of truth for security audits.
To find specific pieces of sensitive data, you run an inspection job on a single resource, like a table or bucket. This job uses an inspection template, which defines what to look for using built-in infoTypes (like US_SOCIAL_SECURITY_NUMBER) or custom infoTypes you create using regular expressions or dictionaries. After a scan, you need to evaluate the findings. To make the inspection more accurate, you can tune it by adjusting the likelihood threshold to filter out low-confidence results and creating hotword rules to only flag data when specific context is present (like the word "SSN" next to a number). For data sources not directly supported, you can export the data to a supported format like CSV in Cloud Storage for inspection.
When sensitive data is found, SDP can apply various techniques to sanitize it without using encryption, which helps preserve the data's usefulness for analysis. Character masking replaces sensitive text with placeholder characters (like ****). Date shifting randomly changes dates within a set range, keeping the order of events intact for time-based analysis. Record suppression can remove entire rows of data from a dataset if they meet certain risky conditions. These methods allow data to be shared or analyzed more safely.
SDP can also protect sensitive information in images using optical character recognition (OCR). The image.redact API can find text, barcodes, or license plates in an image and cover them with opaque boxes before the image is saved or shared. To apply these protections at scale, you integrate SDP into automated data pipelines. For example, you can use native plugins in Cloud Data Fusion to filter or mask data as it flows, or use BigQuery remote functions to run SDP transformations directly within your SQL queries. This ensures data is automatically cleaned before it is stored, exported, or analyzed.
Pseudonymization replaces sensitive data with artificial tokens, but allows you to reverse the process later if needed. This maintains referential integrity, meaning you can still link related data across different systems without exposing the original information. A key technique is Format-Preserving Encryption (FPE), which encrypts data but keeps it in the same format. For instance, an encrypted credit card number still looks like a valid card number, which helps it work with existing databases and applications without requiring changes.
A four-step process flow showing FPE de-identification: a sensitive SSN is detected, a Cloud KMS wrapped AES key is used, the FPE transform replaces it with a same-format token tagged with a surrogate infoType, and a dashed return path shows re-identification with the same key. The animation moves a token dot along the arrows in order, lighting each step as it goes.
These cryptographic transformations rely on keys managed by Cloud Key Management Service (Cloud KMS). The process uses envelope encryption: an encryption key is itself encrypted (or "wrapped") by a Cloud KMS key. This wrapped key is then used by SDP to transform your data, keeping the core encryption keys secure. Reversible transformations like FPE allow for re-identification—getting the original data back—when you have the correct key. During de-identification, you assign a surrogate info type as a marker, so the system knows which data can be reversed later.
Deciding between reversible methods (like FPE) and irreversible methods (like one-way hashing) involves a trade-off. Reversible pseudonymization preserves more analytical utility, letting you join datasets and perform complex analysis, but it requires strict key management. Irreversible methods offer stronger security since the original data cannot be recovered, but they limit how the data can be reused. Your choice depends on your specific need for data analysis, your compliance requirements, and your overall security goals.
A process flow showing envelope encryption: a Cloud KMS key wraps an AES-256 data key, the wrapped key is passed to Sensitive Data Protection, which uses format-preserving encryption to turn an SSN into a same-format token tagged with a surrogate infoType, and a note shows the token can be re-identified later with the correct Cloud KMS key. The animation moves a dot step-by-step along the arrows and highlights each component in order.
Data discovery is a continuous scanning service that builds data profiles of your information across Cloud Storage, BigQuery, and Datastore, giving a high-level view of where sensitive data resides. Data inspection runs an inspection job on a single resource using an inspection template with built-in or custom infoTypes to find specific pieces of sensitive data.
Reversible pseudonymization like FPE preserves analytical utility by allowing you to join datasets and perform complex analysis, but it requires strict key management. Irreversible methods like one-way hashing offer stronger security because the original data cannot be recovered, but they limit how the data can be reused.
Use character masking to replace sensitive text with placeholder characters, preserving the data's usefulness for analysis while hiding the actual values. Use record suppression to remove entire rows of data that meet certain risky conditions, which is more aggressive but may reduce dataset completeness.
An enterprise is building an automated Dataflow pipeline to sanitize structured customer support CSV files before exporting them to BigQuery for analytics. The files contain fixed fields: Customer_ID, SSN, Age, Signup_Date, and a free-text Agent_Notes field.
The security team has established the following requirements:
StudyCerts
StudyCertsSSN by masking all but the last 4 characters.Age into 10-year intervals (e.g., 20-30, 30-40) without incurring content inspection overhead.Signup_Date by a randomized offset to preserve relative time intervals while protecting exact dates.Agent_Notes column with their respective infoType labels.Which Sensitive Data Protection (SDP) configuration should the security team implement?
Create a structured SDP DeidentifyTemplate that defines recordTransformations. Use primitiveTransformation rules with characterMaskConfig for SSN, fixedSizeBucketingConfig for Age, and dateShiftConfig for Signup_Date applied directly to field names. Apply infoTypeTransformations with replaceWithInfoTypeConfig only to the Agent_Notes field, and pass the template ID to the Dataflow pipeline.
Create an SDP InspectTemplate configured with built-in infoTypes for US_SOCIAL_SECURITY_NUMBER, AGE, DATE, and EMAIL_ADDRESS. Apply global infoTypeTransformations with characterMaskConfig, fixedSizeBucketingConfig, and dateShiftConfig across all columns in the Dataflow pipeline.
Configure an unstructured SDP DeidentifyTemplate that utilizes ImageTransformations with redactionColor and ReplaceWithInfoTypeConfig, and embed the raw JSON configuration directly within the Dataflow pipeline worker code.
Configure a BigQuery SQL query pipeline using remote functions hosted on Cloud Run that invokes the DLP API using deterministic encryption (DE) and format-preserving encryption (FFX) for all fields before writing to destination tables.