Secure Dev Test and Production Data Environments

Environment-specific security postures are differentiated sets of identity, network, and data controls applied to development, testing, and production based on purpose and data sensitivity. Production protects live confidential data with the strictest boundaries and narrowest access, while development and testing favor speed and experimentation on minimized, de-identified, or synthetic data. This differentiation follows secure by design (opens in a new tab), which embeds security choices throughout the development lifecycle, and secure by default, which starts every environment from secure settings so baseline protection requires no manual hardening.

1.1Differentiated Postures for Development, Testing, and Production

A differentiated posture limits the blast radius so a compromise or misconfiguration in one environment cannot spread to another. The design applies defense-in-depth (opens in a new tab) at each layer of the application and infrastructure stack, documents the security requirements of each component, and simplifies the system where possible to support resiliency and recovery. A risk assessment (opens in a new tab) catalogs risks and matching controls for internal requirements and external regulations, recognizing that risks differ under shared responsibility with the cloud provider versus on-premises. Production isolates sensitive data in its own service perimeter, enforces multifactor authentication for human access, and denies traffic and privilege by default. Development minimizes collection of personally identifiable information (PII) such as credit card numbers, addresses, and phone numbers, while testing mirrors production controls but operates on classified, tagged, and tokenized copies rather than raw production data. The flow of trust therefore runs from low-sensitivity experimentation in development toward high-sensitivity protection in production, never the reverse.

1.2Principle of Least Privilege and Role-Based Access Control

Role-based access control (RBAC) with least privilege means every user receives only the minimum permissions needed for a role, such as no-access, read-only, or write. Identity and Access Management (IAM) (opens in a new tab) enforces central management of user and group lifecycles, which prevents data theft, preserves model and pipeline integrity (opens in a new tab), and reduces the surface for accidents or attacks. Granularity increases with data sensitivity, so production data lakes, feature stores, and hyperparameters receive tighter role separation than development sandboxes. Access granularity is implemented through complementary controls:

• Map product IAM roles to a user, group, or service account based on project needs, access patterns, or tags for resource-level scoping.
• Set IAM policies with conditions to enforce conditional, attribute-based access to data, models, code, resource settings, and hyperparameters.
• Apply application-level controls, where Cloud Storage uses IAM policies on buckets and managed folders and BigQuery uses IAM roles on datasets plus row-level security and column-level security inside datasets.

Together these controls mean a development analyst depends on broad sandbox grants, while a production operator depends on narrow dataset and column grants that enforce the same least-privilege principle.

1.3Service Account Strategies per Environment

A service account strategy assigns dedicated, least-privilege service accounts to each pipeline and environment rather than sharing one broad identity. The service account for a training pipeline, for example, has permission to read data from only a specific Cloud Storage bucket and to write model artifacts to Model Registry, so compromise of that account cannot reach unrelated buckets or production serving resources. Development accounts may allow broader experimentation, while testing and production accounts are scoped to exact source and destination resources and require provenance checks. IAM Conditions further constrain when a service account can act, such as allowing a pipeline trigger only if the request originates from a trusted Cloud Build trigger. Organization policies provide environment-wide guardrails by disabling external IP allocation to virtual machines (VMs), restricting resource creation to specific geographic locations, and disabling creation of service accounts or their keys in production. Central IAM management plus per-environment accounts creates clear ownership, where each workload depends on its own identity and each identity depends on explicit grants.

1.4Conditional Access and Just-in-Time Elevation

IAM policy configurations tailor who can reach sensitive environments and under what context, which is critical when production holds regulated data and development does not. Privileged Access Manager controls just-in-time, temporary privilege elevation for select principals, and audit logs record that elevated activity for later review. Principal access boundary (PAB) policies limit the resources a principal can access even when other grants exist, preventing privilege sprawl across environments. Additional context-aware controls refine access decisions:

• Extend Identity-Aware Proxy (IAP) access policies to restrict access by IP address and end-user device attributes.
• Combine predefined or custom roles for different user groups through platform access control with IAM.
• Protect interactive development instances with Access Context Manager and Chrome Enterprise Premium, where access is re-evaluated each time a user authenticates.

Elevation therefore flows from a low-privilege base identity to temporary high privilege only after context checks pass, and only for the approved duration.

1.5Perimeters, Data Movement Limits, and Encryption

Network and data perimeters create trust boundaries that stop exfiltration as data flows from development to testing to production. VPC Service Controls defines a service perimeter around Google-managed resources and sensitive data in a Virtual Private Cloud (VPC), and ingress and egress rules control data flow across that perimeter boundary. Firewall rules then restrict inbound and outbound traffic to AI and data resources under a deny-all-by-default policy that explicitly allows only traffic meeting defined criteria. Data movement across privacy boundaries uses governed sharing, where BigQuery sharing and BigQuery data clean rooms provide a security and privacy framework and Looker Action Hub provides a secure cloud path for dashboard delivery. For encryption, Google Cloud encrypts customer data stored at rest by default with no action required, while Cloud Key Management Service (Cloud KMS) with customer-managed encryption keys (CMEKs) gives each sensitive environment control over key generation, storage, and rotation, including software-based or hardware security module (HSM)-protected keys and automated provisioning with Autokey. CMEK-integrated services encrypt data at rest for Vertex AI, Cloud Storage, Cloud SQL, Spanner, and BigQuery, and data in transit is protected by encryption of VM-to-VM traffic within and between peered VPC networks, MACsec over Cloud Interconnect, IPsec over Cloud VPN, and HTTPS with HTTPS load balancers for application calls.

Orchestrate Secure and Repeatable Environment Promotion is the practice of moving the same verified build artifacts from development through test to production through an automated pipeline with security gates at each step. In Google Cloud, Cloud Build creates and tests artifacts, Artifact Registry stores them, Artifact Analysis scans them, Cloud Deploy promotes them in sequence, and Binary Authorization blocks any deployment that lacks required approvals. This chain makes promotion repeatable because infrastructure, code, and configuration follow the same pipeline every time, and secure because only artifacts that pass scans and policy checks can advance.

2.1Automate the Build-to-Promotion Workflow

The automated build-to-promotion workflow is an ordered continuous integration and continuous delivery (CI/CD) sequence that turns a code commit into a production deployment without manual rebuilds. Continuous integration merges code frequently and runs unit and integration tests on every commit, often with a build tool such as Maven, and code that passes becomes a passing build packaged as deployment artifacts. Continuous delivery then copies those same artifacts to preproduction for end-to-end testing and updates production only if those tests pass. The promotion sequence (opens in a new tab) flows in order:

1. Commit and build: A commit to the shared repository triggers a Cloud Build pipeline that compiles code and runs the test suite.
2. Package passing build: Code that passes tests is packaged once, such as a self-executing JAR for Apache Beam on Java or, for Classic Templates, a JSON template file plus JAR plus optional metadata template.
3. Stage in environment store: Artifacts are stored in a bucket or registry in the deployment environment project, such as preproduction, so later stages use the identical files.
4. Test in preproduction: The artifacts deploy to preproduction environments for end-to-end tests without rebuilding.
5. Promote to production: Production is updated only after preproduction checks succeed, either by launching the artifact directly or by letting a scheduler or orchestrator launch it.

Because the same artifact moves forward instead of being rebuilt per environment, security reviews stay narrow, rollbacks can return to a known safe version, and manual errors are removed from patching and deployment.

2.2Enforce Per-Environment Promotion Policies and Approvals

Per-environment promotion policies are rules that define which verified artifacts may deploy to development, test, or production and which approvals are required before production. Cloud Deploy (opens in a new tab) automates delivery to a series of target environments in a defined sequence for Google Kubernetes Engine (GKE), GKE Enterprise, and Cloud Run, with one-click approvals and rollbacks, enterprise security and audit, and built-in delivery metrics. If an attacker compromises the pipeline, the entire application stack can be affected, so an established approval process must be enforced before code reaches production. Binary Authorization enforces that approval at deploy time by attaching configurable signatures called attestations to container images and checking them before the workload starts on GKE or Cloud Run. Binary Authorization uses attestations to prove that a required process was completed, which lets teams enforce that only trusted code runs on target platforms. A policy can require proof for promotion decisions such as:

• The image was created by a specific build system or CI pipeline.
• The image complies with a vulnerability signing policy.
• The image passes the criteria for promotion to the next environment, such as from development to QA.

When the required attestations are missing, deployment is denied even if the image exists in the registry, so promotion depends on both the artifact and its provenance.

2.3Scan Artifacts Continuously Before Production

Continuous artifact scanning is the automated detection of known vulnerabilities in build outputs before they are allowed into production. For containerized workloads, Artifact Analysis (opens in a new tab) automatically scans images when they are uploaded to Artifact Registry by extracting information about system packages in the container. After the initial scan, Artifact Analysis continuously monitors the metadata of scanned images against new information from vulnerability sources. When new or updated vulnerability data arrives, Artifact Analysis updates the metadata of scanned images to keep it current, creates new vulnerability occurrences for new notes, and deletes vulnerability occurrences that are no longer valid. This scanning layer connects to broader supply-chain checks that also belong in the pipeline. The Supply-chain Levels for Software Artifacts (SLSA) framework requires security checks for source code, build process, and code provenance, while Web Security Scanner, part of Security Command Center, scans web applications for common vulnerabilities when integrated into development pipelines. Teams use Assured Open Source Software to consume open-source packages that Google has verified and tested, and they track the number of vulnerabilities identified and remediated, the percentage of deployments that pass scans, and the reduction in incidents caused by software vulnerabilities.

2.4Harden the Delivery Platform and Supply Chain

Hardening the delivery platform means securing the pipeline systems, identities, and artifact stores themselves so promotion cannot be bypassed or tampered with. Cloud Build runs builds on Google Cloud infrastructure with granular Identity and Access Management (IAM) permissions, VPC Service Controls, and isolated and ephemeral build environments. For container images it supports SLSA Level 3 builds, generates authenticated and non-falsifiable build provenance, and exposes security insights that include the SLSA build level, vulnerabilities in build artifacts, and build provenance such as digests of built images, input source locations, build toolchain, build steps, and build duration. Artifact storage and infrastructure definition complete the chain of trust from source to runtime. Artifact Registry (opens in a new tab) centralizes artifacts and dependencies as a universal package manager, where remote repositories cache external sources such as Docker Hub, Maven Central, and PyPI for availability and scanning, and virtual repositories consolidate same-format repositories behind one endpoint with search order that prioritizes private packages to reduce dependency confusion attacks. Artifacts are further protected with access control, VPC Service Controls perimeters, and organization policies, plus Software Bill of Materials (SBOM) generation and Vulnerability Exploitability eXchange (VEX) statements for transparency. The CI/CD pipeline itself depends on correct identities and roles to access the source repository and external services including Google Cloud, while infrastructure pipelines that deploy infrastructure, build containers, and create data pipelines use Terraform templates to build system infrastructure and Open Policy Agent to check platform conformance, which provides auditability, traceability, and repeatability.

2.5Apply Promotion Patterns for Data Pipelines

Data pipeline promotion applies the same artifact-forward principle to batch and streaming Dataflow jobs, but the deployment action differs by pipeline type. A passing Dataflow build produces environment-portable artifacts that continuous delivery copies across deployment environments for testing before production launch. Batch promotion can launch a new Dataflow job directly from the artifact, or other systems can launch the artifact on demand, such as Managed Service for Apache Airflow running batch jobs within a workflow or Cloud Scheduler scheduling batch jobs. Streaming promotion is more complex because an existing streaming job may need to be replaced or updated, and when an in-place update is not possible or not chosen, teams coordinate multiple Dataflow jobs in parallel to minimize business disruption.

Choose direct launch when each batch run is independent and repeatable from the same artifact, and choose parallel-job coordination when streaming continuity requires the old and new Dataflow pipeline workflows (opens in a new tab) to overlap until the new job is verified.

Data governance and compliance across environments is the practice of applying the same classification, encryption, and lifecycle rules in development, testing, and production while varying how much sensitive data each environment can see. A central governance layer discovers data, monitors data flows from collection to storage to transformation, and enforces access so production retains full-fidelity data under strict controls and non-production uses masked, tokenized, or synthetic substitutes. In Google Cloud, this layer is built from Sensitive Data Protection (opens in a new tab) for discovery and de-identification, Knowledge Catalog for discovery and lineage, Cloud Key Management Service (Cloud KMS) for key control, Virtual Private Cloud Service Controls (VPC Service Controls) and Assured Workloads for boundaries, and logging services for auditability.

3.1Classify and Discover Data Consistently

Knowledge Catalog is the organization-wide platform that centrally discovers, manages, monitors, and governs data and AI artifacts across data platforms. The catalog connects to sources including BigQuery, data lakes, and open-format tables, and it enriches entries with business glossaries, aspects and aspect types, data quality checks, data profiles, and feature and model metadata. Inspection results from Sensitive Data Protection flow from BigQuery scan jobs into Knowledge Catalog, so classification tags created once in production propagate as policy inputs to development and testing projects. Because the catalog also manages column-level lineage and unifies governance across BigLake and formats like Iceberg and Delta Lake, every environment inherits the same definition of what is confidential, including credit card numbers, addresses, phone numbers, and other personally identifiable information (PII).

3.2Control Encryption and Keys by Compliance Need

Encryption at rest and in transit is applied consistently, while control over keys varies by regulatory need. By default, Google Cloud encrypts stored data and encrypts traffic between virtual machines (VMs), over Cloud Interconnect with MACsec, over Cloud VPN with IPsec, and to applications with TLS. When compliance requires customer control, customer-managed encryption keys (CMEKs) in Cloud KMS (opens in a new tab) encrypt data at rest for integrated services including BigQuery, Cloud Storage, Cloud SQL, Spanner, and Vertex AI Platform, with options for hardware-backed keys or external keys.

The choice depends on ownership and tradeoff. Default encryption minimizes operational overhead but does not satisfy an obligation to manage keys. CMEK shifts responsibility to application teams to follow mandated key lifecycles, which the security team enforces with CMEK organization policies that block non-compliant resources. Tokenization preserves analytical utility while removing direct exposure, at the cost of building and operating an ingestion pipeline that tokenizes before persisting to BigQuery.

3.3Separate Production Fidelity from Development Safety

Separation of development from production means production keeps raw values under fine-grained access while development and testing use minimized substitutes. Data minimization requires teams to not collect, keep, or use data beyond strict business need, and where possible to use synthetic or fully anonymized data for building and testing. Sensitive Data Protection (opens in a new tab) applies utility-preserving transformations before data flows from the production perimeter to lower environments:

• Pseudonymization replaces direct identifiers with reversible aliases tied to a separate mapping.
• De-identification removes or obscures PII while retaining format for joins and testing.
• Generalization such as bucketing reduces precision, for example grouping exact ages into ranges.
• Tokenization replaces sensitive cardholder or financial values with non-sensitive tokens at ingestion.
• Synthesis generates fully artificial rows that preserve distributions without containing real subjects.

In production, BigQuery enforces row-level security and column-level security with policy tags, Identity and Access Management (IAM) roles on datasets, and dedicated least-privilege service accounts that can read only a specific Cloud Storage bucket and write only to Model Registry. In development, the same policy tags resolve to masked or tokenized columns, and cross-boundary needs are met with BigQuery sharing and BigQuery data clean rooms that allow analysis across privacy boundaries without copying raw tables. Organization policies reinforce the boundary by restricting resource locations, disabling external IP allocation, and disabling service account key creation globally or per folder.

3.4Manage Lifecycle, Lineage, and Audit Trails

Lifecycle, lineage, and audit controls ensure that a confidential record can be traced from source to use and retained only as long as policy allows. Cloud Storage lifecycle rules transition or expire objects to protect resources from prolonged access, while BigQuery, Dataproc Metastore, and lakehouse catalogs record where each table and column originated. The governed flow runs in order:

1. Categorize: Sensitive Data Protection scans and classifies source data from Google Cloud or external ingest.
2. De-identify: The pipeline applies masking, tokenization, or column-level encryption before storage.
3. Store in perimeter: Confidential data lands in BigQuery inside a VPC Service Controls perimeter (opens in a new tab), with encrypted values and wrapped keys held in separate tables for externally encrypted ingest.
4. Re-identify on demand: Authorized production identities reverse the protection for approved analysis, with every access logged.

Lineage for models, datasets, and pipeline artifacts flows from Vertex AI Pipelines and Vertex AI ML Metadata into Knowledge Catalog for debugging and root-cause analysis, and Model Registry versions each stored model. Auditability depends on Cloud Audit Logs capturing application programming interface (API) calls and administrative actions, Cloud Logging and Cloud Monitoring capturing pipeline logs and metrics without logging PII, and Observability Analytics answering who accessed or modified data and when. Assured Workloads (opens in a new tab) establishes data boundaries for FedRAMP, Criminal Justice Information Services (CJIS), and other regulated workloads, and VPC Service Controls configures ingress and egress rules so data cannot move from the production perimeter to development except through approved de-identification paths.

Key Facts

• Production isolates live sensitive data with deny-by-default perimeters, multifactor authentication, and least-privilege IAM, while development and testing use minimized, tokenized, or synthetic copies.
• Dedicated least-privilege service accounts per pipeline and environment prevent a compromise in one workload from reaching unrelated buckets or production serving resources.
• The same verified artifact moves from commit to preproduction to production without rebuilding, and Binary Authorization blocks deployment on GKE or Cloud Run when required attestations are missing.
• Artifact Analysis scans images at upload to Artifact Registry and continuously updates findings, while Binary Authorization enforces policy at deployment time.
• Default encryption uses Google-managed keys, while customer-managed encryption keys (CMEKs) with Cloud HSM or Cloud External Key Manager are required when auditors demand lifecycle ownership, HSM protection, or external key storage.