Protecting Data with DLP and Encryption

Protecting sensitive data and preventing data loss combines discovery and transformation of sensitive data, identity and perimeter controls on data services, secret lifecycle management, and metadata hardening. Sensitive Data Protection (opens in a new tab) finds and protects data where it is stored, while BigQuery, Cloud Storage, and Cloud SQL add fine-grained and network controls. Secret Manager and Compute Engine metadata controls close credential and instance-identity paths that attackers often abuse.

1.1Configuring Sensitive Data Protection for Discovery and De-identification

Configuring Sensitive Data Protection (SDP) defines how sensitive data is found with profiles and inspection jobs and how it is protected with reversible or irreversible transformations. Cloud Data Loss Prevention (Cloud DLP) is now part of Sensitive Data Protection, while the API name remains the Cloud Data Loss Prevention API (DLP API). SDP scans data where it is stored, so data does not leave its region of origin during inspection or profiling.

Discovery profiling continuously scans at the organization, folder, or project level and produces data profiles with sensitivity and data risk levels. Inspection jobs perform detailed on-demand scans of specific Cloud Storage buckets, BigQuery tables, or Datastore kinds and return individual findings with quotes and exact locations. Synchronous content methods inspect data inline through the API before it is stored, which blocks or transforms sensitive data before it enters backend storage. Discovery acts as the source of truth for where sensitive data lives, while an inspection job is used when investigation needs exact locations after discovery reports a high free-text score or multiple infoTypes. Job triggers automate inspection jobs on a schedule so repositories are rescanned as data changes.

Inspection templates in SDP are reusable configurations that persist selected infoTypes, optional rulesets, and a confidence threshold. A template is valid only when at least one infoType is selected, and the same template can be reused across Cloud Storage, BigQuery, and Firestore in Datastore mode. Templates are regional resources, and a template used for profiling must be in the same region as the data to be profiled. Built-in infoTypes cover more than 100 categories such as PERSON_NAME, PHONE_NUMBER, US_SOCIAL_SECURITY_NUMBER, and DATE_OF_BIRTH, while custom infoTypes extend detection with regular expressions, dictionaries and word lists, and stored infoTypes. Raising the confidence threshold reduces noise but can miss weaker matches, hotword rules raise likelihood when a nearby word such as patient supports a match, and exclusion rules omit overlapping matches such as a DOMAIN_NAME inside an email address.

SDP divides de-identification work into two transformation scopes that control how findings flow into protection. recordTransformations apply a configuration per field and are designed for structured or tabular data, while infoTypeTransformations inspect for sensitive data and transform each finding and are designed for unstructured or free-text data. Non-cryptographic methods such as masking, redaction, bucketing, and date shifting are generally one-way, so the sanitized replica cannot be reversed to recover originals. Cryptographic pseudonymization preserves utility with tokens: Format-Preserving Encryption (FPE) keeps original format and length, deterministic encryption always produces the same token for the same input with the same key and configuration to preserve joins and counts, and cryptographic hashing is one-way and cannot be decrypted. The transformation uses a Cloud Key Management Service (Cloud KMS) wrapped key in a kmsWrapped block with a wrappedKey and cryptoKeyName, where SDP calls Cloud KMS to unwrap the key, encrypts in memory, and writes only the token to output. The surrogate infoType labels the token for later re-identification, and only data encrypted with a reversible algorithm can be restored through projects.content.reidentify with the same wrapped key, surrogate infoType, and template. Text redaction returns findings organized by InfoType, Likelihood, and Offset or sanitized output where sentence structure remains, image redaction detects text with object character recognition (OCR) and objects such as OBJECT_TYPE/BARCODE and obscures them with opaque rectangles, and statistical k-anonymity requires quasi-identifiers for each person to match at least k minus 1 others.

1.2Restricting Access to BigQuery, Cloud Storage, and Cloud SQL Datastores

Restricting access to Google Cloud data services combines identity-based access with perimeter and fine-grained controls to prevent unauthorized exposure. BigQuery is the analytical warehouse with dataset, table, column, and row controls, Cloud Storage is the object store reachable from inside and outside Google Cloud, and Cloud SQL holds sensitive transactional data. Identity and Access Management (IAM) decides which principal has which role on which resource, while VPC Service Controls decides whether the request context allows it to cross a security perimeter.

Object protection for Cloud Storage uses two independent boundaries. Uniform bucket-level access makes the IAM policy the single authority for buckets and objects, so a principal can upload or read data only with appropriate permissions checked on every access attempt. VPC Service Controls forms a security boundary around resources such as Cloud Storage and BigQuery, and Access Context Manager access levels permit only designated principals based on attributes such as IP address and identity. Traffic flows from the requester to VPC Service Controls for boundary evaluation before it reaches the bucket or object, so copying data outside without a matching access level is blocked. A single large perimeter around all sensitive projects gives the most effective exfiltration protection with lowest overhead, while multiple perimeters isolate high-trust from de-identified data, regulated data, or independent tenants.

BigQuery fine-grained access control restricts visibility below the dataset level. Organizations contain projects, projects contain datasets, and tables inherit permissions from the dataset that contains them. Table-level access grants a user, group, or service account access to a specific table or view without granting the entire dataset. Column-level security uses policy tags organized in a taxonomy created in Data Catalog, where a user must have dataset access plus the Data Catalog Fine-Grained Reader role on the relevant tag to read protected column data. A common tiered taxonomy defines 3_Confidential for credit card numbers, 2_Private for personally identifiable information (PII) such as first name, and 1_Sensitive for non-public data such as credit limit. Query behavior fails closed: a user without the tag grant who runs SELECT * receives an error naming the required tag and can retry with SELECT * EXCEPT (ssn). Policy tags apply only to leaf fields of STRUCT or RECORD columns, require GoogleSQL, and clearing a tag requires setting schema policyTags.names to []. Row-level security filters data so a principal sees only rows that satisfy a qualifying condition evaluated at query time, and an authorized view is authorized to read its source dataset so users can query view results without access to underlying source data. There is a hard limit of 2500 authorized resources per dataset, including authorized views, datasets, and functions.

Cloud SQL secure connectivity removes public network paths, requires encrypted transit, and allows only authorized identities to perform work. Private IP connectivity means clients reach the instance through a Virtual Private Cloud (VPC) network instead of the public internet, so traffic never leaves private boundaries. Public access is restricted by preventing public IP assignment, auditing existing public IPs, and allowlisting hybrid addresses when needed, with Cloud VPN or Cloud Interconnect protecting hybrid paths. Enforced SSL/TLS encryption means every incoming database connection must use cryptographic protection with minimum TLS 1.2 or higher, and sensitive instances pair transport encryption with customer-managed encryption keys (CMEKs) so data at rest is encrypted under customer-controlled keys. Cloud IAM database authentication grants access only through least-privilege roles with separation of duties and restricted service-account key creation.

1.3Securing Secrets with Secret Manager Lifecycle and Access Controls

Securing secrets with Google Secret Manager controls a secret from creation through version replacement, rotation, and expiration. A secret is the named container such as projects/PROJECT_ID/locations/LOCATION/secrets/SECRET_ID, and each secret version is an immutable copy of the credential value inside it. Secret values can be in any format but must not be larger than 64 KiB, and the first value uploaded at creation becomes the first version.

Secret versions move between lifecycle states that support safe rotation. Versions appear in the ENABLED state after creation or rotation and in the DISABLED state after a disable operation. A disabled version cannot be used by future access requests until re-enabled, while other versions of the same secret are unaffected. The workflow depends on the new version becoming ENABLED while the prior version is retained in DISABLED form for rollback. For specialized use, a regional secret can be created with a fixed secretType such as CLOUD_SQL_DB_CREDENTIALS, which tells Secret Manager that rotation must also update the linked Cloud SQL instance. The secret type cannot be added or changed on an existing secret.

A rotation schedule is a start time plus a recurring period that defines when the value must be replaced. When rotation runs for a Cloud SQL credential secret, Secret Manager first creates a new version and then attempts to update credentials in the Cloud SQL instance, and only on success does rotation status change to Enabled. An event-driven design extends this flow where a Pub/Sub notification carries the rotation event to a Cloud Functions handler that generates the new credential and propagates it before the old version is disabled. An expiration policy is a time limit after which the entire secret is no longer usable, where rotation maintains freshness during the lifetime and expiration guarantees termination at the end of it. Use expire-time with an absolute RFC 3339 timestamp such as 2100-01-01T09:00:00-05:00 when the secret must end on a known calendar date, and use time-to-live (TTL) with a relative duration such as 86400s when the secret must live only a short lifetime from creation. Expiration applies to the secret container while rotation creates new versions inside it, so applications that depend on the secret must refresh it before expiration.

Granular IAM grants access per secret to a specific service account rather than broad project access. A binding attaches roles/secretmanager.secretAccessor on a single secret resource to one member, so only that workload identity can read that version. Authentication scopes are a second gate for Compute Engine and Google Kubernetes Engine (GKE) workloads: the instance or node pool must be provisioned with the cloud-platform OAuth scope or calls fail with Request had insufficient authentication scopes even when IAM is correct. Traffic flows from application code through the client library to the Secret Manager API only after both checks pass. Enabling access follows an ordered dependency: first enable the Secret Manager API, then grant secret-level IAM to the workload service account, then provision the host scope at creation or with gcloud compute instances set-service-account. Customer-managed encryption replaces Google default encryption when compliance requires customer control, where the customer creates a key ring and symmetric key in Cloud KMS and the Secret Manager service agent must hold roles/cloudkms.cryptoKeyEncrypterDecrypter on the key. The key location must match the secret location, CMEK must be selected at creation, automatic replication lets Google replicate within its footprint while user-managed replication pins replicas to customer-selected regions, and Cloud Audit Logs plus Cloud KMS Data Access logs record administration, access, and Encrypt and Decrypt calls. A service perimeter encloses selected projects so a request from inside to Secret Manager inside the same perimeter is permitted while a copy directed outside is denied, and Private Google Access lets instances without external IPs reach Google APIs without traversing the public internet.

1.4Protecting and Hardening Compute Instance Metadata

Protecting compute instance metadata limits who can manage virtual machine (VM) metadata and which processes can read it. Every Compute Engine instance can query the metadata server automatically without extra authorization, and the request and response never leave the physical host that runs the VM. Any process that can reach the metadata query URL can read all values, including custom metadata, certificates, and keys, so both outside management and inside guest access must be restricted.

The metadata trust boundary is the physical host that runs the VM. Traffic between the VM and the metadata server never leaves that host, but inside the guest there is no per-user check by default. A workload can request service account tokens from the metadata server, and any local process with network access to the query URL receives the same full set of values. Generic service account names reduce exposure because viewing an allow policy can expose emails: jenkins@deployment-project-123.iam.gserviceaccount.com reveals project ID and software type, while deployer@deployment-project-123.iam.gserviceaccount.com avoids disclosing the software. If a service account is granted access to a less-controlled project, its email becomes visible more widely.

IAM controls metadata management from outside the VM at project, zone, instance, and service-account levels. Querying metadata from inside requires only permission to connect to the instance, while viewing or changing metadata from console, CLI, or REST API requires Compute Engine and service-account permissions. Project metadata requires compute.projects.get to view and compute.projects.get plus compute.projects.setCommonInstanceMetadata to change, zonal metadata requires compute.instanceSettings.get plus compute.instanceSettings.update to change, and instance metadata requires compute.instances.get plus compute.instances.setMetadata to change. When VMs use service accounts, callers also need iam.serviceAccounts.actAs. Granting Service Account Token Creator at project level lets a user impersonate any service account in that project, so manage access on each individual service account instead of inheriting broad access. Shell access to a VM with a privileged service account is equivalent to the ability to act as that service account because a shell user can query the metadata server for tokens. OS Login enforces a stronger binding on Linux by requiring OS Login permission plus iam.serviceAccounts.actAs on every connection, while metadata-based SSH keys and Windows passwords check metadata access plus iam.serviceAccounts.actAs only when the key is published or credential requested. In-guest access is unrestricted by default, so even nobody or LocalService can obtain tokens; the fix is enforced in the guest firewall with Linux iptables rules using --uid-owner and --gid-owner or Windows rules customized with Set-NetFirewallSecurityFilter. Metadata endpoints expose the same entries over HTTP at http://metadata.google.internal/computeMetadata/v1 and http://169.254.169.254/computeMetadata/v1, host maintenance can interrupt the server for less than one second with HTTP Error 503 so applications should retry, and storage is bounded at 512 KB combined total, 128 bytes per key, and 256 KB per value. Server-Side Request Forgery (SSRF) is the primary external path: traffic flows from the attacker to a vulnerable web application on the VM, then from that application to the metadata service, then back to the attacker. Secure access means only on-instance code can reach 169.254.169.254 while the internet cannot reach the instance, following the order of first configuring Private-IP-only addressing or restricting firewall ingress, then creating a Google Cloud Armor policy, then enabling logging and threat detection.

Managing encryption at rest, in transit, and in use defines who owns key material, where cryptographic operations run, and how keys rotate, revoke, and protect storage, transit, and memory. Cloud Key Management Service (opens in a new tab) centralizes customer-managed and external keys, while lifecycle policies and Confidential Computing extend protection to object retention and data in use. The choice of Google-managed, customer-managed, customer-supplied, or external keys determines control, availability dependencies, and compliance posture.

2.1Selecting Default Encryption, CMEK, and External Key Management

Selecting default encryption, customer-managed encryption keys (CMEK), and external key management fixes the ownership boundary for data at rest. Google default encryption is automatic protection that requires no key configuration, where data moving to Google Cloud is protected in transit with Transport Layer Security (TLS) and data is protected at rest once stored. Keys for this baseline are Google-owned and Google-managed keys that Google creates, stores, protects, and operates under FIPS 140-2 compliance. Workloads receive baseline confidentiality without managing a key ring, key versions, or rotation schedules.

Customer-managed encryption keys (CMEK) are customer-owned keys in Cloud Key Management Service (Cloud KMS) that replace Google key control for data at rest. With CMEK, the customer owns a CryptoKey in Cloud KMS, and the storage or compute service depends on that key for every encrypt and decrypt operation. The customer controls generation, use, rotation, and destruction but also assumes responsibility for key availability and recovery. Key administration can live in a dedicated CMEK project separate from the data project, while service agents receive only encrypt and decrypt rights on the key.

Cloud External Key Manager (Cloud EKM) is a Cloud KMS key type where key material lives in an external key manager outside Google Cloud. The Google service stores only encrypted data, so no decryption can succeed without a request flowing from the Google service to the external manager and back. Key Access Justifications (KAJ) adds per-request governance by requiring Google systems to attach a justification code to every operation on an enrolled key, where a key policy can allow or deny access based on that code and audit logs record the code. Common codes include CUSTOMER_INITIATED_SUPPORT, GOOGLE_INITIATED_SERVICE, and THIRD_PARTY_DATA_REQUEST, while REASON_NOT_EXPECTED and REASON_UNSPECIFIED indicate missing context rather than verified purpose.

2.2Choosing Software, Hardware, and External Protection Levels

Protection level in Cloud KMS is the setting that fixes where cryptographic operations run for a CryptoKey version. The level is set in versionTemplate.protectionLevel at creation and cannot be changed afterward, so isolation must be selected before the key protects data. Changing isolation later requires creating a new key with a different level.

SOFTWARE protection means operations run in software in the Cloud KMS service environment and provides baseline protection acceptable for lower-risk use cases. Hardware Security Module (HSM) protection means operations run inside a Google-managed hardware security module and provides higher security generally recommended for higher security requirements. EXTERNAL and EXTERNAL_VPC mean operations run in an external key manager over the internet or over a Virtual Private Cloud (VPC) network, with key material staying outside Google and support limited to symmetric encryption and asymmetric signing. KeyRing location controls replication and failure isolation: a multiregion or global KeyRing replicates metadata and avoids a single-region point of failure, while a single-region KeyRing keeps keys local. Cloud SQL requires the Cloud KMS key version and the instance to be in the same region, and a multiregion or global key causes the instance create request to fail. For EKM keys, resilience depends on the EkmConnection having redundant resolvers, because an unreachable external key can suspend a Cloud SQL instance for hours. CMEK integration requires the per-product service account to hold the cloudkms.cryptoKeyEncrypterDecrypter role on the key, and every cryptographic call flows from the using service to the Cloud KMS API so quota and key state affect performance and uptime.

2.3Creating and Rotating CMEK and EKM Keys with Revocation

Creating and managing encryption keys for CMEK and EKM covers revocation by disabling, destruction by crypto-shredding, external key coordination, and automatic rotation. Revocation by disabling is the reversible act of turning off a key version so services can no longer use it. Google Cloud services that use CMEK call Cloud KMS on each data operation, so when Cloud KMS reports the key as disabled or destroyed, dependent data becomes inaccessible. Disablement can take up to three hours to propagate to dependent services.

Scheduled destruction is the irreversible lifecycle that permanently removes a key version's ability to perform operations for crypto-shredding. The lifecycle moves through ordered states with a built-in recovery window:

1. Request destruction: The version moves from ENABLED or DISABLED to SCHEDULED FOR DESTRUCTION for its configured duration.
2. Restoration window: The version remains restorable only while SCHEDULED FOR DESTRUCTION.
3. Destroyed state: After the duration passes, the version moves to DESTROYED and logical deletion begins without customer recovery.
4. Residual retention: Key material can remain in Google systems for up to 45 days while deletion completes.

Destruction permanently disables the version and stops billing for it, while deletion only removes the record from listings and cannot substitute for destruction. Deleted CryptoKey names cannot be reused, and destroying all active versions does not stop automatic rotation unless rotation is explicitly disabled. Cloud EKM keys are pointers to material that stays outside Google Cloud, where symmetric encryption flows first from Cloud KMS to its internal material for initial encryption, then to the external manager for a second wrapping layer, then back as ciphertext that requires both materials to decrypt. Manually managed keys require the administrator to maintain rotation and destruction on both sides without modifying external material, while coordinated external keys use an EKM connection with Cloud KMS management mode so creation, rotation, and destruction initiated in Cloud KMS are forwarded to the external manager. A destroyed Cloud EKM version cannot be recreated even with the same URI or key path. Automatic rotation creates a new primary version on a schedule so new data encryption keys are wrapped with the new version, but only symmetric ENCRYPT_DECRYPT keys with GOOGLE_SYMMETRIC_ENCRYPTION and Google-generated material can rotate automatically. Rotation changes only the primary version for new encryptions, while prior enabled versions remain required to decrypt existing data, with service-specific behavior such as Spanner re-encrypting its data encryption key, Cloud Storage buckets applying the new version only to newly written objects, and BigQuery tables keeping the creation version by default.

2.4Applying Envelope Encryption and Tokenization to Use Cases

Applying encryption methods to use cases determines whether plaintext is protected inside the customer environment, which key model owns the key encryption key, and how transit and use are secured. Client-side envelope encryption protects payloads inside the customer environment before transmission to Google Cloud. The application encrypts sensitive fields locally with a data encryption key (DEK) using Google Tink, then wraps that DEK with a symmetric CMEK key encryption key (KEK) managed in Cloud KMS and generated in Cloud HSM. Data flows from the customer application to BigQuery through an ordered envelope sequence: first the application obtains a fresh DEK, then encrypts the field before data leaves the customer trust boundary, next calls Cloud HSM to wrap the DEK with the KEK, and finally writes ciphertext to a BigQuery data table and the wrapped DEK to a separate key table. Field-level tokenization with Sensitive Data Protection replaces personally identifiable information with tokens before storage, where an ingestion pipeline reads raw data from Cloud Storage, passes it through detectors and a Dataflow pipeline, and writes tokenized output to a BigQuery warehouse.

Key management models for data at rest define who owns the key encryption key, where material lives, and who controls rotation and audit. Google Cloud uses envelope encryption where a KEK wraps data encryption keys that directly encrypt data. Google-owned default encryption is automatic with keys in Keystore using AES-256, Cloud Key Management Service (Cloud KMS) CMEKs are customer-owned KEKs where the service agent performs operations transparently, Cloud KMS Autokey automates provisioning with AES-256 GCM keys co-located with the resource and rotated yearly, Cloud EKM keeps keys outside Google with justification-gated use, and Customer-Supplied Encryption Keys (CSEK) supply material per request with Google holding it only in memory for Cloud Storage. Default protection for data in transit covers traffic inside Google's production network, where virtual machine (VM)-to-VM traffic within a VPC and between peered VPCs is encrypted and traffic between Google services uses Application Layer Transport Security (ALTS). Additional protection is required across trust boundaries: Cloud VPN uses IPsec, Cloud Interconnect can use MACsec because Interconnect traffic is not encrypted by default, Apigee supports TLS and mutual TLS for API ingress, Cloud Service Mesh provides mutual TLS for containerized applications, and Confidential Computing with a Confidential VM uses hardware-based memory encryption so data stays encrypted while processed.

2.5Configuring Conditional Lifecycle Rules for Cloud Storage Objects

Conditional lifecycle rules in Cloud Storage are automated bucket rules that pair conditions with actions applied only to matching objects. The condition defines scope, such as objects that reached a required age, objects in a specific storage class that should move to cheaper archival storage, or objects with custom metadata marking sensitivity. The action then transitions storage class or deletes objects whose retention period ended, and Cloud Storage evaluates the rules automatically as new data arrives.

Version-aware lifecycle management extends that logic to versioned objects when Object Versioning is enabled. One object name can have one live generation plus zero or more archived generations preserved after overwrite or delete. A version-aware rule can keep the live object available while expiring old archived generations after a defined period. Deleting a specific archived generation requires the bucket name, object name, and generation number together, so the live generation is unchanged when the rule targets an archived generation. Retention Policy with Bucket Lock and Soft Delete are bucket-level guardrails beside lifecycle rules, where a Retention Policy prevents deletion or replacement until its duration expires and Soft Delete keeps an object for a recovery window after deletion. Archival transitions move objects to Nearline, Coldline, and Archive tiers where the object remains stored and encrypted but with an infrequent-access profile. Encryption does not change on transition, but objects protected by CMEK depend on Cloud KMS key availability on every API call: if the key is disabled, destroyed, or denied, the archived object becomes unreadable even though the transition succeeded, while deletion plus key destruction provides strong de-risking through crypto-shredding.

2.6Enabling Confidential Computing with Attested Execution Environments

Enabling Confidential Computing adds encryption in use inside a hardware-based trusted execution environment (TEE). A TEE is a secure area inside the processor that protects the confidentiality and integrity of code and data loaded inside it. Data flows from encrypted storage over an encrypted connection into encrypted memory, is decrypted only inside the processor boundary, and results leave encrypted for storage or transmission. Four hardware-backed properties work together: runtime encryption keeps TEE data encrypted in memory, isolation blocks software access from outside except over defined interfaces, attestation produces a cryptographically signed report for verification before releasing keys or data, and data lifecycle security extends protection into processing.

Confidential VM is a virtual machine type that uses hardware-based memory encryption to protect data and applications while in use, with supported hardware including AMD SEV, AMD SEV-SNP, Intel TDX, Arm CCA, IBM Z, IBM LinuxONE, and Nvidia Confidential GPU. With Confidential VM, the cloud stack and administrators, BIOS and firmware, and host operating system and hypervisor move outside the trust boundary, while the guest administrator, guest operating system, applications, and data remain inside. Confidential GKE Nodes run containers as Confidential VMs so container memory is hardware-encrypted, with configuration at cluster, node-pool, or workload level. Confidential Space creates a secure area within a VM with the highest isolation, removing even the guest administrator from the boundary and keeping only measured and attested guest components plus data inside. Attestation proves an instance is legitimate before data is released: the CPU chip produces a signed report covering bootloader, hardware, firmware, and boot-time events, the report flows to an attestation service for verification, and only after validation does the owner release keys or data. End-to-end designs combine this gate with split trust requiring authorization from two independent key management systems and federated patterns where collaborators send only aggregated model updates to a TEE server.

Securing AI workloads protects training data, prompts, models, and agent interactions from leakage, tampering, and extraction across the full lifecycle. Google Kubernetes Engine (opens in a new tab) and Vertex AI controls combine data minimization, encryption, perimeters, identity isolation, and provenance so only governed data trains models and only verified models serve traffic. Agent platform controls extend the same boundaries to agent identities, tools, and data sources.

3.1Protecting AI Data, Pipelines, Storage, and Model Serving

Protecting AI data, pipelines, storage, and model serving prevents training data, prompts, and models from leaking or being misused by accident. The control plane combines governance of data flows, encryption of pipelines and storage, and integrity checks on models from collection through training to inference. Training data carries the highest risk for bias and leakage, so minimization, de-identification, lineage, and audit logging apply across the full lifecycle.

Data minimization collects, keeps, and uses only data strictly necessary for the business goal, because extra personally identifiable information increases privacy risk and can create biased models. Pipelines preserve utility while reducing exposure with pseudonymization, de-identification, generalization such as bucketing exact values into ranges, and use of synthetic or fully anonymized data. A data labeling job adds informative tags to raw data to create structured training data, while Cloud Storage lifecycle controls limit how long training datasets, artifacts, and stored prompts stay available. Sensitive Data Protection discovers and classifies sensitive data with inspection jobs at organization, folder, or project level, where results flow into BigQuery data profiles and into Knowledge Catalog for governance. Protection is enforced before training and at serving time: pipelines run checks to find PII before it reaches training, and Model Armor enforces a minimum safety threshold and applies a Sensitive Data Protection DeidentifyTemplate that redacts or masks sensitive information before the response reaches the user.

Knowledge Catalog is the organization-wide governance platform that discovers, manages, monitors, and governs data and AI artifacts. Data flows from systems such as BigQuery, Cloud Storage, and Pub/Sub into the catalog for inventory, then flows back as governed datasets, feature metadata, and model artifacts. Lineage including column-level lineage in BigQuery tracks sources and transformations, validation checks plus anomaly detection flag poisoning and drift, and differential privacy adds noise during analysis or training so individual records are hard to identify while patterns remain learnable. Secure pipelines move data through an ordered flow where each phase inherits classification, encryption, and access policy: first ingestion validates formats, inspects PII, and captures quality profiles; then processing links lineage, flags outliers, and monitors features; finally storage and serving expires temporary data, versions artifacts, and logs access. Network controls prevent exfiltration between phases with VPC Service Controls perimeters validated first in dry-run mode, deny-all firewall policy, and cross-boundary sharing through Analytics Hub, data clean rooms, and authorized views instead of copying raw tables. Vertex AI Model Registry versions each trained model and links it to training parameters, dataset version, code version, and evaluation metrics. The signing flow runs from build to deployment in a signed to verified to deployed lifecycle, where Cloud Build signs the artifact and verification runs before promotion. Binary Authorization verifies signatures for containerized models, while direct endpoint models require custom hash checks. Hardening defends against inversion and extraction by reducing memorization, placing registry and data inside a perimeter, serving through private endpoints based on Private Service Connect, fronting public inference with Apigee for validation and throttling, and monitoring for sequential probing with rate limiting and traffic splitting.

3.2Defining Security for IaaS-Hosted and PaaS-Hosted Training Models

Defining security for Infrastructure as a Service (IaaS)-hosted and Platform as a Service (PaaS)-hosted training models matches network, hardware, identity, and artifact controls to who manages the training infrastructure. In PaaS managed training Google secures the underlying compute while the customer controls perimeters, keys, and identities around it. In IaaS self-managed training the customer additionally hardens the operating system, cluster, and runtime that hold data and weights.

PaaS isolation in Vertex AI blocks exfiltration from Google-managed training compute attached to the customer Virtual Private Cloud (VPC). Vertex AI custom training jobs, pipelines, managed datasets, Model Registry, and Workbench notebooks run as managed resources attached to the customer VPC. The training project, Cloud Storage training buckets, and related APIs are placed inside the same VPC Service Controls perimeter, and any call that tries to leave the perimeter is denied unless an explicit ingress or egress rule allows it. Managed training supports private-only communication through a Private Service Connect (PSC) interface and private Internet Protocol (IP) addressing, where traffic flows from the custom training job to Cloud Storage and Vertex AI datasets through the PSC interface inside the perimeter without public exposure. Workbench control prevents notebooks from becoming an exfiltration path by blocking public IPs, requiring attachment to a designated private subnet, and enabling Private Google Access. Customer-managed encryption keys must be enabled per resource type and do not automatically inherit from the bucket to the model, so each layer including datasets, Feature Store, models, endpoints, jobs, and Workbench disks needs its own key assignment.

Hardened IaaS training isolates self-managed training on Compute Engine and GKE from unauthorized infrastructure access. Isolation starts by placing training in dedicated projects or VPC networks so traffic flows from training virtual machines to storage and services only through private paths. Shielded VMs protect the boot chain by authenticating boot components during startup, where Secure Boot blocks rootkits and bootkits that would steal training data or tamper with weights. Confidential Computing as Confidential VMs protects data while in use in memory, which default encryption at rest and in transit does not cover. GKE hardening uses private nodes without public IPs, network policies, pod security controls, trusted images scanned with Artifact Analysis, GKE Sandbox for kernel-level isolation, separate node pools for different sensitivity levels, and OS Login for Secure Shell (SSH) identity. Remediation for a noncompliant boot configuration follows a fixed order: first create a new node pool with Secure Boot enabled, then migrate workloads until the old pool is empty, and only after migration delete the original pool. Least-privilege IAM gives pipelines only needed storage and registry access, while Artifact Registry with Binary Authorization ensures clusters run only images that pass checks. Graphics processing units (GPUs) provide general-purpose parallel processing for varied training, while Tensor Processing Units (TPUs) are custom accelerators optimized for large-model training and inference.

3.3Enforcing Security Controls for Gemini Enterprise Agent Platform

Enforcing security controls for the Gemini Enterprise Agent Platform protects agent identities, runtime infrastructure, and AI data as one connected system. The platform governs which users and agents can invoke which agents, tools, and data sources, isolates training and prediction inside private boundaries, and classifies and encrypts data from ingestion through prediction. Google secures the physical infrastructure and platform foundation while the customer configures access, networks, and data controls.

Agent authorization uses Identity and Access Management (IAM) as the authorization system for all Google Cloud resources. Human users obtain capabilities through platform roles: the Gemini Enterprise Agent Platform User role grants access to most platform capabilities, while the Administrator role grants full control. Agents act with their own service-account identities rather than borrowing broad user permissions, where the Identity function manages agent service accounts and Policy Enforcement applies IAM and Model Armor policies. When a Gemini Enterprise app calls a custom agent, the request flows from the app to the agent endpoint using the Discovery Engine Service Agent identity, and the network path and IAM check depend on hosting location. Granular enforcement grants IAM roles to groups rather than individuals, requires Cloud Identity, Google Workspace, or federated identities hardened with single sign-on and 2-step verification, and resolves @agent_name references through the registry to an Agent Runtime resource URI or Agent-to-Agent (A2A) URL invoked with the Discovery Engine Service Agent identity. Custom agents on Agent Runtime travel over the Google network as an internal API call with built-in roles, custom agents on Cloud Run travel to the registered run.app URL through Google Front End ingress and require Cloud Run Invoker, and custom agents on GKE require public DNS resolution to an external IP with a managed load balancer and Identity-Aware Proxy (IAP).

Agent infrastructure isolation keeps training and runtime inside private boundaries. Training runs on managed services or in dedicated projects or VPCs, with Shielded VMs or Confidential Computing, trusted scanned images, and Model Registry for versioned artifacts. Network boundary control combines VPC firewall rules that filter traffic with VPC Service Controls perimeters that contain data movement. Data flows from custom training jobs to authorized services through the Private Service Connect interface without traversing the public internet, and data flows from clients inside a VPC to models through private prediction endpoints protected by a perimeter. IAM is not applied directly to the private prediction endpoint; the target service behind the endpoint uses IAM to manage access to models. Data governance classifies and encrypts training data, prompts, and outputs with Sensitive Data Protection discovery, pseudonymization, de-identification, and bucketing, while Cloud KMS keys encrypt AI data at rest for Agent Platform, Cloud Storage, BigQuery, Cloud SQL, and Spanner. Only Gemini Enterprise Agent Platform supports all four controls of data residency at rest, CMEK, VPC-SC, and Access Transparency, compared with partial support in RAG Engine and Vector Search. Monitoring combines platform audit logs, Cloud Trace, Cloud Logging, Cloud Monitoring, and OpenTelemetry telemetry for execution paths and token usage, with Security Command Center integration for anomalous activity and Workforce Identity Pool administration restricted through Privileged Access Manager.

Key Facts

• Sensitive Data Protection discovery profiling produces data profiles continuously while inspection jobs return detailed findings with quotes and exact locations, and profiling templates must be in the same region as the profiled data.
• recordTransformations protect per-field structured data while infoTypeTransformations transform only detected findings in unstructured text, and only reversible Format-Preserving Encryption or deterministic encryption can be restored with the same Cloud KMS wrapped key and template.
• BigQuery column-level security requires dataset access plus the Data Catalog Fine-Grained Reader role on the policy tag, and authorized views are limited to 2500 authorized resources per dataset.
• Secret Manager secret versions are immutable, limited to 64 KiB, and move between ENABLED and DISABLED states, while Compute Engine metadata is limited to 512 KB combined with 128 bytes per key and 256 KB per value.
• A Cloud KMS protection level is fixed at creation with SOFTWARE running in software and HSM running inside a hardware security module, and destroying a key version permanently removes decryption ability after the scheduled destruction period.