professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Transformational de-identification modifies sensitive records such as personally identifiable information (PII) so organizations can analyze data without exposing original identities. Google Cloud provides these capabilities through Sensitive Data Protection, which replaces sensitive data points with protected tokens or generalized values. Two-way tokenization methods, such as format-preserving encryption (FPE) and deterministic encryption (DE), replace raw values with encrypted values using a key. These two-way techniques preserve referential integrity, which allows analytical systems to join related records across tables without exposing the underlying plaintext. In contrast, one-way tokenization uses cryptographic hashing to generate irreversible tokens, making it ideal for joining records when original values never need to be decrypted.
| Technique | Reversibility | Format Preserved | Primary Use Case |
|---|---|---|---|
| Format-preserving encryption (FPE) | Reversible (Two-way) | Yes | Downstream legacy systems requiring specific lengths and character sets |
| Deterministic encryption (DE) | Reversible (Two-way) | No | High-security analytical joins where original field format does not matter |
| Cryptographic hashing | Irreversible (One-way) | No | Analytical matching and joins where plaintext data must never be recovered |
Non-cryptographic transformations obscure or generalize data values to reduce re-identification risks without using cryptographic keys. Masking replaces sensitive characters with a fixed symbol, such as an asterisk, hiding identifiers while partially displaying non-sensitive fragments. Bucketing groups exact numerical or demographic values into generalized ranges, such as converting specific ages into ten-year brackets. Date shifting moves recorded timestamps by a consistent, pseudorandom offset per individual, preserving the time intervals between events while masking actual calendar dates. Choosing between these techniques requires balancing data utility against privacy preservation, because aggressive masking or bucketing reduces the statistical detail available to analysts.
Exam tip: Choose format-preserving encryption (FPE) when downstream legacy applications or database constraints require data to maintain its original string length and character set, such as a ten-digit telephone number.
Scalable de-identification architectures use automated data processing pipelines to clean data before it reaches analytical storage. In this workflow, raw data streams into Cloud Dataflow, which processes batch or streaming records by executing transformation rules defined in Sensitive Data Protection templates. These templates decouple transformation logic from application code, allowing security teams to manage policies centrally and reuse them across multiple pipelines. Cryptographic techniques depend on a token encryption key managed through Cloud Key Management Service (Cloud KMS) as a wrapped key. Administrators regularly rotate these encryption keys and assign separate keys to distinct data fields to prevent a single compromised key from exposing all datasets.
Raw data from Cloud Storage flows into Cloud Dataflow, which applies Sensitive Data Protection inspection and de-identification templates to tokenize PII before the tokenized data is stored in BigQuery. Cloud KMS supplies the wrapped token encryption key, and all processing resources sit inside a VPC Service Controls perimeter.
Granular access controls in analytical warehouses restrict data visibility based on user identity, column sensitivity, and row attributes. BigQuery secures sensitive attributes through column-level security using policy tags managed in Dataplex Universal Catalog (formerly Data Catalog). When a policy tag is applied to a column, only users with specific Identity and Access Management (IAM) roles can view the unmasked data. BigQuery also supports row-level access policies, which filter query results based on user identity, and authorized views, which grant query access to aggregated results without exposing the underlying base tables. Dynamic data masking works alongside column-level security to let authorized users query non-sensitive columns while automatically showing masked or hashed values for protected columns.
Exam tip: Column-level security completely blocks unauthorized users from querying a column, whereas dynamic data masking allows queries to run while obscuring the sensitive values.
Enterprise data governance combines automated metadata tracking with virtual perimeters to prevent unauthorized data access and exfiltration. Administrators configure VPC Service Controls (VPC SC) to build a security perimeter around resources like BigQuery and Cloud Storage, blocking unauthorized data transfers to external environments. Security policies can also enforce context-aware access, limiting service account and user operations to verified corporate IP addresses and managed devices. To maintain continuous governance across large data estates, administrators implement the following automated practices:
Regulatory compliance requires organizations to track how sensitive data moves through the cloud and monitor every user interaction. Google Cloud captures administrative and data-access operations using Cloud Audit Logs and Access Transparency logs, providing an immutable record of user and service actions. Dataplex tracks data lineage automatically, recording where datasets originate, what transformations run against them, and which downstream tables or dashboards consume them. This lineage visibility allows compliance teams to trace the flow of sensitive data, verify that protection policies were applied during processing, and detect anomalous data access patterns.
Automated detection identifies and categorizes sensitive data across structured databases, data warehouses, and object storage. Sensitive Data Protection uses built-in infoTypes, which are detectors configured to identify specific data categories such as credit card numbers, Social Security numbers, passport IDs, and medical records. These predefined detectors analyze data patterns using regular expressions, character context, and mathematical checksum algorithms to reduce false positives. When organizations handle proprietary internal formats, such as custom customer identification numbers or employee IDs, administrators create custom infoTypes using regular expressions, word dictionaries, or stored lookup tables.
Google Cloud provides two complementary scanning mechanisms to discover and monitor sensitive data at scale:
Exam tip: Sensitive data profiling continuously monitors risk posture across organizations and projects, whereas on-demand inspection evaluates specific tables, columns, or files to locate sensitive records down to individual cells.
Inspection templates standardize detection rules across continuous data ingestion pipelines and batch scanning jobs. These templates define the exact infoTypes to detect, the likelihood thresholds required to trigger a match, and the maximum number of findings to record per scan. Security teams manage access to these templates using IAM roles, ensuring that developers and data engineers can apply standardized classification rules without altering security configurations. When integrated into ingestion pipelines with Cloud Dataflow or Cloud Storage, inspection templates ensure that all incoming data is classified consistently before it is committed to analytical storage.
Column-level security completely blocks unauthorized users from querying a protected column, whereas dynamic data masking allows queries to run while automatically displaying masked or hashed values for sensitive fields. Both features utilize policy tags managed in Dataplex Universal Catalog to enforce access controls based on user identity.
Format-preserving encryption (FPE) should be selected when downstream legacy systems or database constraints require data to maintain its original string length and character set. Deterministic encryption (DE) is better suited for high-security analytical joins where the original field format does not need to be preserved.
Sensitive data profiling automatically scans entire organizations, folders, and projects to generate high-level risk profiles and monitor overall risk posture. On-demand inspection conducts targeted, deep scans on specific Cloud Storage buckets, BigQuery tables, or relational databases to locate sensitive data down to individual cells and columns.
Your organization manages an enterprise analytics warehouse in BigQuery containing tables with sensitive personally identifiable information (PII), such as government identification numbers and contact details. You need to implement a scalable, fine-grained access control architecture that enforces column-level security.
The solution must satisfy the following requirements:
Which strategy should you implement?
Declare primary and foreign key constraints on the sensitive columns in BigQuery, and configure Cloud Armor authorization policies based on VM tags to restrict query results at the network layer.
Create individual authorized views for every analyst group that exclude sensitive columns, remove all dataset permissions from the underlying tables, and manage column access solely through authorized view permissions.
Create a hierarchical taxonomy with policy tags, assign policy tags to sensitive columns via schema annotations, enforce access control on the taxonomy, and grant compliance officers the Data Catalog Fine-Grained Reader role on the appropriate policy tags while granting analysts standard BigQuery dataset permissions.
Define policy tags within a flat taxonomy, assign the policy tags to columns using standard GoogleSQL CREATE TABLE DDL statements, and grant compliance officers the BigQuery Data Viewer role while granting analysts the Data Catalog Policy Tag Admin role.
StudyCerts
StudyCerts