Build-Time and Runtime Secret Injection

GCP-specific secret management tools for each phase separate build-time provisioning from runtime injection. Build-time provisioning makes a secret available to an ephemeral build step, while runtime injection makes the live secret value available to a running workload. Secret Manager (opens in a new tab) is the central store for both phases, Cloud Build (opens in a new tab) consumes secrets during the build, and running services on Google Kubernetes Engine (GKE), Cloud Run, or Vertex AI Pipelines fetch secrets after deployment.

1.1Build-Time Secret Provisioning With Cloud Build and Secret Manager

Build-time secret provisioning with Cloud Build and Secret Manager delivers a secret version to specific build steps without storing the value in source control or the build configuration. The flow runs from Secret Manager to the build step to the consuming process: the secret and version are stored in Secret Manager, the build configuration maps that version to a local environment variable name, and only steps that declare access receive the value in memory during execution. UTF-8 secrets use the environment-variable pattern, while non-UTF-8 secrets require a separate binary-safe handling path.

Configuration uses the availableSecrets block after all build steps in cloudbuild.yaml or cloudbuild.json. The ordered setup is:

1. Declare the mapping: Add availableSecrets.secretManager with pairs of versionName, the full path such as projects/PROJECT/secrets/SECRET/versions/VERSION, and env, the local variable name. Substitution variables can be used in versionName.
2. Scope access per step: In each step that needs the secret, set entrypoint to bash, add secretEnv listing the allowed env names, and start args with -c so the following string runs as a bash command.
3. Consume without persisting: Use the variable inside scripts, Docker authentication, or API calls during that step, so the value resolves at build time and is not committed to the repository.

The build service account must hold Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) on the specific secret before the build runs, and both the Cloud Build and Secret Manager APIs must be enabled.

1.2Runtime Secret Injection for Deployed Workloads on GKE and Serverless

Runtime secret injection for deployed workloads keeps application secrets out of deployment artifacts and Cloud Deploy (opens in a new tab) pipelines. The running Pod, Cloud Run service, or Vertex AI pipeline task authenticates to the Secret Manager API and reads a specific secret version, so secrets are generated, managed, and rotated outside the delivery pipeline. Provisioning creates or updates the secret in Secret Manager before the application deploys, so the workload never starts without its required version.

The preferred pattern is direct API access from application code using client libraries, REST, or gRPC with Application Default Credentials. Direct access avoids expanding the trust boundary to the filesystem or process environment, where directory-traversal vulnerabilities can read secret files and where debug endpoints or logging dependencies can leak environment variables. Product integrations still support environment variables or volume mounts when direct calls are impractical, including native Cloud Run secret bindings and the Secrets Store CSI driver that mounts Secret Manager (opens in a new tab) secrets as volumes for GKE and attached clusters with Workload Identity authentication.

The two phases differ in where the secret resolves and how the value reaches the workload:

Additional metadata such as secret version or environment name like dev, staging, or prod can be passed as ordinary non-sensitive environment variables, while the secret material itself stays in Secret Manager.

1.3IAM Authentication and Network Controls for Build and Runtime

Identity and Access Management (IAM) enforcement for build and runtime separates who can use a secret during the build from who can use it after deployment. Both phases follow least privilege but depend on different identities: the Cloud Build service account needs accessor rights during the build, while the GKE workload, Cloud Run service, or Vertex AI pipeline service account needs accessor rights at runtime. Authentication should use Application Default Credentials from the platform instead of exporting a service account key, with local development using gcloud auth application-default login, Compute Engine and Cloud Run using the instance metadata server, GKE using Workload Identity, and Amazon Web Services or Microsoft Azure using workload identity federation.

Project, secret, and network boundaries prevent one compromise from crossing environments:

• Segment staging and production into separate projects so project-level IAM bindings isolate environments and quotas are enforced independently.
• Choose a curated role with minimal permissions or create a custom role, and prefer secret-level IAM bindings or IAM Conditions to limit a principal to only the necessary subset of secrets.
• Use IAM Recommender to identify over-privileged bindings and time-based IAM Conditions as a reversible alternative to expiring production secrets.
• Add defense in depth with a Virtual Private Cloud (VPC) Service Controls perimeter around the Secret Manager API, the constraints/iam.allowedPolicyMemberDomains organization policy to limit which domains can appear in secret IAM policies, and data access logs for AccessSecretVersion enabled at the folder or organization level.

When secret material is synced to another datastore such as Kubernetes Secrets, which are not secure by default without encryption, that store must be evaluated explicitly for whether it expands access beyond the Secret Manager IAM policy, whether access remains auditable, and whether it meets data-at-rest encryption and regionalization requirements.

1.4Versioning Rotation and Lifecycle Safeguards in Secret Manager

Secret lifecycle controls in Secret Manager prevent outages and limit the blast radius of a leak across both build and runtime use. References should pin a secret by version number rather than the latest alias, so configuration can be validated through existing release processes and rolled back by redeploying the prior version number if a new version is broken. Versions move through ordered states instead of immediate deletion: first disable a version, which blocks use like destroy but remains reversible, then wait to confirm no lingering build or runtime dependencies, and only then destroy or delete.

Replication and expiration choices control availability and permanence. The automatic replication policy is the default unless location requirements enforced by constraints/gcp.resourceLocations apply, while regional secrets are chosen when strict data residency requires at-rest, in-use, and in-transit guarantees inside a specific geography. Do not set expiration on production secrets unless irreversible deletion is intended; expiration fits automated cleanup of temporary environments. Periodic rotation limits the impact if a value has leaked, prevents principals who no longer require access from continuing to use a previously accessed value, and reduces outage risk from stale credentials. Monitoring uses Cloud Asset Inventory exports of secret metadata to find secrets and detect non-conformance for rotation, encryption, and location, while capacity planning estimates peak AccessSecretVersion usage for concurrent deploys or autoscaling thundering herds and requests a quota increase if the project cannot absorb the burst.

Security and operational characteristics of injection methods differ because build-time embedding couples the secret lifecycle to the artifact lifecycle, while runtime fetching decouples them. Build-time injection embeds values into images or deployment artifacts during the build, so anyone who can read the artifact can recover the secret. Runtime injection keeps secrets in Secret Manager and fetches decrypted values only when the workload runs, so the same immutable image can run in development, staging, and production with different secrets resolved at startup.

2.1Security Posture of Build-Time Secret Embedding in Artifacts

Security posture of build-time secret embedding is weak because sensitive values such as database credentials, API keys, certificates, or passwords persist in every copy of the artifact. Once baked into source code, .env files, container build configurations, or deployment artifacts, a secret persists in Cloud Deploy (opens in a new tab) manifests, Artifact Registry copies, layer history, build provenance, build logs, dashboards, and Software Bill of Materials (SBOM) metadata. Cloud Deploy pipelines should not manage secrets this way because injecting application secrets into deployment artifacts introduces security risks, and secrets should be generated, managed, and rotated outside the scope of Cloud Deploy.

Build-time values also break least privilege because the build service account, artifact readers, vulnerability scanners, and deployment approvers all gain indirect access to the secret. An attacker who compromises a repository, modifies pipeline configuration to pull a different base image, or publishes a typosquatted package can capture embedded values during the build. Because rotation requires rebuilding, re-scanning with Artifact Analysis, re-attesting with Binary Authorization, and redeploying the image, teams delay rotation and widen the window of exposure after a leak. Kubernetes Secrets do not fix this by default because Kubernetes Secrets are not secure by default without encryption, and syncing secret material into them expands access without necessarily providing auditability or compliant encryption.

2.2Security Posture of Runtime Secret Injection With Secret Manager

Security posture of runtime secret injection is stronger because the secret stays in Secret Manager, a fully managed multi-region service, and the workload fetches the decrypted value at execution. Applications and pipelines access named secret values based on permissions granted with Identity and Access Management (IAM), and every interaction can create an audit trail for forensics and compliance. Authentication flows from the workload identity to the Secret Manager API using Workload Identity on GKE, the instance metadata server on Compute Engine and Cloud Run, Application Default Credentials locally, or workload identity federation on other clouds, which avoids exporting an additional service account key.

Runtime access enforces least privilege at the individual secret level by granting Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) only on the specific secrets each microservice requires, optionally narrowed with IAM Conditions and project separation for staging versus production. High-value payloads can be wrapped with customer-managed encryption keys (CMEK) in Cloud KMS, verified with data integrity checksums on add and access, replicated automatically for availability or stored as regional secrets for data residency, and isolated with a VPC Service Controls perimeter and the constraints/iam.allowedPolicyMemberDomains organization policy. Automated rotation publishes a SECRET_ROTATE notification to a Pub/Sub topic when the interval starts, then a Cloud Run functions or Cloud Run service reads the notification, generates a new value, adds the new version, and destroys the retired version, while Cloud Audit Logs record version creation, destruction, and payload access for alerting in Google SecOps. Direct API access is preferred over passing secrets through the filesystem or environment variables, because filesystem secrets raise the severity of directory traversal and environment secrets can leak through debug endpoints or logging dependencies.

2.3Operational Impact on Deployment Speed Rollback and Consistency

Operational impact on deployment speed, rollback, and consistency comes from separating code releases from secret changes while still using the release process to control which secret version is active. The application is configured with a specific secret version that is read on startup, and updates to version numbers are deployed through existing release processes with environment variables carrying non-sensitive metadata such as secret version or application environment. Pinning to a version number allows the configuration to be validated and rolled back with standard deployment controls, whereas following the latest alias can leave a workload unable to start if a new version is bad. Cloud Deploy preserves one-click approvals, rollbacks, and per-environment policy checks for the deployment itself without becoming the system of record for secret values.

Consistent environments depend on provisioning order and lifecycle discipline. If a feature deployment provisions infrastructure, the secret is created or updated in Secret Manager as part of provisioning before the application is deployed, so the workload never starts without its required version. Secret versions are disabled before they are destroyed or deleted, which creates a reversible state that blocks use while lingering dependencies are found, and expiration is reserved for temporary environments rather than production because expiration irreversibly deletes data. Periodic rotation limits the impact of a leak, revokes use by people who no longer need access, and reduces outage risk, while Cloud Asset Inventory monitors rotation, encryption, and location conformance. Peak usage must be planned because concurrent deploys or autoscaling can create a thundering herd of AccessSecretVersion calls, so projects need enough Secret Manager quota and data access logs enabled at the folder or organization level to observe access patterns.

Secure secret lifecycle management is the design of continuous integration and continuous delivery (CI/CD) pipelines so secrets are created, stored, rotated, and revoked outside the deployment system and fetched only when needed. In Google Cloud, Secret Manager stores API keys, passwords, and certificates, while Cloud Build builds artifacts and Cloud Deploy delivers them to GKE or Cloud Run without embedding secret values. The pipeline depends on Secret Manager for values, on IAM for permission checks, and on Cloud Logging and Pub/Sub for audit and rotation events across development, staging, and production.

3.1Segregated Secret Access With Least Privilege Across Pipeline Stages

Segregated secret access means every pipeline stage, microservice, and human identity can access only the specific secrets it requires for its job. Access to the Secret Manager API is protected by IAM, so architecture starts by segmenting applications and environments into separate projects, which isolates environments with project-level IAM bindings and enforces quotas independently. Within a project that holds secrets for many services, access is narrowed to individual secrets rather than the whole project. This prevents over-granting such as granting project-level access to Cloud Storage when one bucket would suffice.

Confidentiality and integrity models set the direction of allowed flows. To maintain confidentiality, the Bell-LaPadula model requires that data flows from lower to equal confidentiality only: the deployment pipeline (opens in a new tab) must not consume input artifacts of higher confidentiality and must not write data to a resource of lower confidentiality. To maintain integrity, the Biba model requires the opposite direction for trust: the deployment pipeline must not consume input artifacts of lower integrity and must not write data to a resource of higher integrity. Pipeline category is set by first inventorying the data, applications, and resources the pipeline touches and categorizing them, then using the highest confidentiality or integrity category touched as the pipeline category, except when create-without-read allows a lower category.

Practical least-privilege controls combine to enforce those flows:

• Separate development, staging, and production into different projects and grant pipeline service accounts (opens in a new tab) access to individual resources, never to whole projects or the organization.
• Apply IAM policy bindings directly to specific secrets, granting Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) only on required secrets, and choose a curated role with minimal permissions or a custom role when needed.
• Add IAM Conditions for attribute-based access, such as allowing a service account to trigger a pipeline only from a trusted Cloud Build trigger, and use tags to map roles to users, groups, or service accounts.
• Limit standing privilege with principal access boundary (PAB) policies and Privileged Access Manager for just-in-time, temporary elevation, and use IAM Recommender to find over-privileged bindings.
• Authenticate workloads without long-lived keys by using Application Default Credentials from the instance metadata server on Compute Engine and Cloud Run, Workload Identity on GKE, and Workload Identity Federation on Amazon Web Services or Microsoft Azure.

3.2Minimized Secret Exposure Duration Across Build and Runtime

Minimized secret exposure duration means a secret value exists in memory or in transit for the shortest time and in the fewest places, with build-time embedding avoided in favor of runtime fetching. Because injecting application secrets into deployment artifacts introduces security risks, secrets consumed by applications deployed with Cloud Deploy must be generated, managed, and rotated outside the scope of Cloud Deploy. The deployment process provisions infrastructure first and creates or updates the secret in Secret Manager (opens in a new tab) before the application deploys, then the running application fetches the decrypted value when it needs it.

Runtime fetching is preferred because it keeps the pipeline from reading data it does not need and keeps artifacts free of long-lived values. Secrets should not pass through the filesystem or environment variables when direct API calls are possible, because directory traversal can expose files and debug endpoints or logging dependencies can leak environment details. When secret material is synced to another store such as Kubernetes Secrets, which are not secure by default without encryption, that store must be evaluated for whether it expands access, whether access can be audited, and whether it meets encryption-at-rest and regionalization requirements.

Automated rotation further shortens exposure by replacing values on a schedule and destroying retired versions. A rotation schedule on Secret Manager follows an ordered workflow:

1. Publish: When the rotation interval starts, Secret Manager publishes a SECRET_ROTATE notification to a designated Pub/Sub topic.
2. Trigger: A Cloud Run functions or Cloud Run service subscribed to the topic reads the notification.
3. Replace: The function generates a new secret value and adds the new version to Secret Manager.
4. Retire: The function destroys the retired version so only the current version remains accessible.

Payloads are protected with customer-managed encryption keys (CMEK) in Cloud Key Management Service (Cloud KMS) to control, rotate, or revoke wrapping keys, with data integrity checksums when adding and accessing versions, and with the automatic replication policy unless location constraints require pinned regions enforced by constraints/gcp.resourceLocations.

3.3Audit Logging and Monitoring for Secret Access Events

Audit logging for secret lifecycle management is the complete, retainable record of who created, accessed, rotated, or destroyed each secret version in every environment. Cloud Audit Logs on Secret Manager track secret version creation, destruction, and payload access events, and those logs route to Google SecOps to alert on suspicious access such as reads outside standard hours or attempts to read unapproved secrets. Admin Activity audit logs, Data Access audit logs, and pipeline execution logs for the same pipeline run must be correlated, because Cloud Logging retains Data Access logs for a shorter default period than Admin Activity logs and pipeline systems may discard logs even sooner.

Aligned retention ensures evidence is available for forensics and compliance across development, staging, and production. Retention for Data Access audit logs is customized with custom log buckets or a custom sink routes logs to long-term storage when defaults are insufficient or misaligned. Pipeline attribution is enriched so API activity maps to a specific run by passing the pipeline run ID in system parameters on API requests, using a custom User-Agent that embeds the run ID, logging API requests performed by each run, and recording any returned operation ID in the CI/CD logs.

Continuous monitoring extends beyond access events to artifact and posture signals. Google Cloud console dashboards review vulnerabilities in build artifacts, build provenance with image digests, source locations, toolchain, steps and duration, and Software Bill of Materials (SBOM) dependency lists, with maturity assessed by the Supply-chain Levels for Software Artifacts (SLSA) framework. Cloud Build supports SLSA Level 3 builds for container images with authenticated, non-falsifiable provenance from isolated and ephemeral build environments, while Binary Authorization enforces that only attested images promote from development to quality assurance to production.

3.4Isolated Environments and Contained Exfiltration Paths in Pipelines

Isolated pipeline architecture limits the blast radius so a compromised build step or deployed workload cannot move data across environments. The scope of each deployment pipeline covers a small number of resources, cross-project access is avoided, and pipelines that span multiple projects or environments are split into individually secured stages. The pipeline and its infrastructure are treated like a production system, holding the CI/CD software, tools such as Terraform, kubectl, and Docker, and the operating system to the same integrity standards as the managed resources, verifying tool authenticity, rebuilding infrastructure from scratch, and running builds on shielded virtual machines (VMs).

Exfiltration can be direct or indirect, and each path requires a different boundary. In a direct path, the pipeline service account itself has permission to read confidential data from Cloud Storage, BigQuery, or other locations and a bad actor uses its access token to copy data out. In an indirect path, the pipeline has permission to create compute resources with attached service accounts, so a bad actor compromises the pipeline to deploy malicious code and that code steals data from inside Google Cloud. Direct risk is reduced by revoking unneeded read access and unneeded write access even to non-confidential destinations, and by blocking internet access or restricting connections to an allow-listed set of network locations, which is especially important for moderately or highly confidential pipelines with access to data or key material.

Virtual Private Cloud (VPC) Service Controls contains the indirect path by restricting the set of resources and APIs accessible from within certain Google Cloud projects. The protected resources sit inside the service perimeter and the pipeline sits outside, so traffic flows from the pipeline to the perimeter only through an ingress rule that should allow only the pipeline IP addresses and only the services the pipeline truly needs. Combined with dedicated projects or VPCs for training and deployment, private endpoints with Private Service Connect, and Binary Authorization promotion checks, this perimeter ensures compromised deployments cannot reach unapproved APIs or exfiltrate data to external locations.

Key Facts

• Cloud Build provisions build-time secrets through an availableSecrets.secretManager mapping of versionName to env combined with per-step secretEnv, with the build service account granted roles/secretmanager.secretAccessor on the specific secret.
• Runtime injection fetches secrets from Secret Manager via direct API calls, Cloud Run bindings, or the Secrets Store CSI driver with Workload Identity, keeping secret values out of images, manifests, and Cloud Deploy pipelines.
• Build-time embedding persists secrets in images, layers, logs, and SBOM metadata and requires rebuild, rescan, and redeploy to rotate, while runtime fetching lets the same image run across environments with versioned configuration and scheduled rotation.
• Secret references should pin a version number rather than latest, disable versions before destroying them, and avoid expiration on production secrets.
• VPC Service Controls perimeters, secret-level IAM with Conditions, Application Default Credentials without exported keys, and Cloud Audit Logs for version and payload access enforce least privilege and auditability across development, staging, and production.