Continuous integration (CI) with Cloud Build

Design and Optimize Build Configurations and Artifacts

Continuous integration with Cloud Build relies on a cloudbuild.yaml configuration file that defines the automated pipeline. This file contains steps—individual commands or actions—and builders, which are container images that run those steps. Designing the file efficiently means organizing steps in a logical order, using official builders like gcr.io/cloud-builders/kubectl for Kubernetes deployments, and specifying artifacts such as binaries or container images that the pipeline produces.

Caching strategies reduce build time by reusing layers from previous builds. For example, you can cache a node_modules directory or Maven dependencies so that the pipeline does not re-download them each time. After the build finishes, artifacts are stored in Artifact Registry for versioned container images and language packages, or in Cloud Storage for general output like logs and deployment manifests. The choice depends on the artifact type and the need for versioning or secure access.

Build triggers automate the pipeline by listening for repository events such as a push to a branch (e.g., production) or the creation of a pull request. Triggers connect to source repositories like GitHub, Bitbucket, and Cloud Source Repositories. For advanced workflows, you can chain pipelines: a successful CI build in one repository can push a generated artifact (like a Kubernetes manifest) to another repository’s branch, which then triggers a separate delivery pipeline, creating a full CI/CD chain.

Security is part of the build configuration. Sensitive data such as API keys or deployment tokens must not appear in the source code. Instead, you use Secret Manager to store them. In the cloudbuild.yaml, you reference secrets using availableSecrets and secretEnv fields, which inject them as environment variables only into specific steps. This keeps credentials secure and audited without exposing them in logs or code.

Configure and Manage Cloud Build Triggers

Cloud Build triggers start builds automatically in response to changes in source code or external events. They are the foundation of a continuous integration pipeline. Three trigger types exist:

• Repository event triggers respond to push events from repositories like Cloud Source Repositories, GitHub, or Bitbucket. You filter by specific branches or tags using branch/tag filters.
• Schedule-based triggers use cron expressions to run builds at particular times, useful for nightly builds or periodic security scans.
• Webhook triggers let external systems send HTTP requests to start builds, integrating with third-party tools.

When configuring a trigger, substitution variables pass dynamic values such as commit hashes or branch names into the build configuration. This makes pipelines reusable across environments.

Proper IAM permissions are required for triggers to work. The service account used by the trigger must have the cloudbuild.builds.create permission. Without it, you get a “Permission denied” error. The Cloud Build service agent needs the roles/cloudbuild.serviceAgent role and the roles/serviceusage.serviceUsageConsumer role to access project resources. For private repositories, you may need to configure SSH keys or API tokens as build steps. The Cloud Build service account (roles/cloudbuild.builds.builder) includes permissions for Artifact Registry and other Google Cloud services.

Troubleshooting common trigger issues:

• A “Couldn’t read commit” error means the branch you specified does not exist; check spelling.
• For Pub/Sub triggers, “Request is prohibited by organization’s policy” indicates the Pub/Sub API is restricted by VPC Service Controls; you may need to temporarily remove the service from the perimeter.
• When accessing private repositories, manually set up authorization credentials because the git credential helper is removed after cloning.
• SSH authorization errors require proper SSH key setup for private GitHub repositories.
• Network issues in private pools can cause “No route to host” errors if the IP range 192.168.10.0/24 conflicts with your VPC; choose a different range.
• Timeout errors may mean builds cannot reach external resources; ensure external IPs are enabled for private pools when needed.

Integrate Security and Compliance in CI Pipelines

Infrastructure as Code Validation

Infrastructure as Code (IaC) validation lets DevOps teams inspect Terraform files for security issues before deployment. By integrating Security Command Center with Cloud Build, the pipeline automatically detects violations of organizational policies. This shift-left validation compares resource definitions against active security detectors. Implementing these checks early in the CI/CD pipeline prevents misconfigured resources from reaching production.

IAM Roles and Least Privilege

Securing CI pipelines requires configuring IAM roles under the principle of least privilege. The legacy Cloud Build service account often had overly broad permissions. To reduce risk, switch to user-specified service accounts that have only the actions they need. For example, grant the Security Posture Shift-Left Validator role to enforce policy validation without exposing administrative access.

Secure Secret Management

Sensitive data such as API tokens or private SSH keys must not be hardcoded in build configurations or code. Use Secret Manager to store credentials centrally. Assign the Secret Manager Secret Accessor role to the service account that needs to retrieve them during runtime. This centralization ensures compliance with security standards while enabling secure connections to external repositories.

Network Security and Private Pools

To comply with strict policies like VPC Service Controls, run builds inside a Cloud Build private pool. These private pools run workloads within a Google-managed peered VPC network, providing network-level isolation from the public internet. Setting up a private pool requires a private service connection to allow secure communication with internal resources, such as private GKE clusters. Enforcing private connections protects build artifacts and maintains compliance throughout the deployment lifecycle.