A CI/CD pipeline is a set of steps that automatically builds, tests, and deploys code every time a developer pushes changes. When designing a pipeline, the team decides what happens at each stage: code is pulled from the repository, compiled or packaged, tested, and then deployed to an environment. The architecture must match the type of workload—application code, infrastructure definitions, or machine learning models—because each has different requirements. For example, an ML pipeline needs stages for data validation and model training that a standard application pipeline does not require.
Different patterns serve different needs. A linear pipeline runs each stage one after another and works well when stages have clear dependencies. A parallel pipeline runs independent stages at the same time, which speeds up the overall process. A triggered pipeline starts automatically when events occur, such as a code push or a scheduled time. The team chooses a pattern based on how quickly they need feedback and how complex their release process is.
Cloud Build is Google's service that runs builds in the cloud. It takes source code, runs the steps defined in a configuration file, and produces artifacts like containers or deployable packages. The configuration file lives in the repository alongside the code, so the pipeline definition stays versioned with the software it builds. For infrastructure, teams often use Terraform or Infrastructure Manager to apply configuration changes automatically. For machine learning workloads, Vertex AI Pipelines provides a way to orchestrate ML workflows using Kubeflow or Tekton components.
Continuous testing means running tests automatically every time code changes. Unit tests run first to check individual functions, then integration tests verify that components work together, and finally end-to-end tests simulate real user scenarios. The pipeline stops and notifies the team if any test fails, preventing bad code from reaching production. Test results are stored and tracked so the team can see trends over time, such as whether the code is becoming more reliable or if specific areas are breaking frequently.
Pipeline configuration lives as code in the repository, which brings the same benefits as application code: version control, review processes, and history tracking. Changes to the pipeline go through code review before being applied, reducing the risk of accidental misconfiguration. The configuration defines which steps run, what environment variables are set, and which resources are used at each stage.
Secrets such as API keys, passwords, and tokens must never appear in plain text in configuration files. Secret Manager stores secrets securely, and the pipeline retrieves them at runtime when needed. Access to secrets is controlled through IAM, ensuring only the pipeline service account can read them. This separation means secrets are not baked into artifacts and can be rotated without changing the pipeline code.
The pipeline itself must be protected because a compromised pipeline can deploy malicious code. Access to pipeline settings is restricted to authorized users through IAM roles. The service account that runs the pipeline should have only the permissions it needs, following the principle of least privilege. Audit logs track who made changes to the pipeline and when, supporting accountability and compliance requirements.
Supply chain security focuses on ensuring that the software being deployed comes from trusted sources and has not been tampered with. Artifact Registry stores container images and other artifacts with vulnerability scanning enabled. The pipeline can verify image signatures before deployment, ensuring that only images built from approved source code are released. This protects against attacks that try to inject malicious code into dependencies or build processes.
Professional Cloud DevOps Engineer
Prepare and test your skills
Prepare and test your skills
Each workload type has different requirements: a standard application pipeline focuses on compiling, testing, and deploying code, while an ML pipeline needs additional stages for data validation and model training, and infrastructure pipelines often use tools like Terraform or Infrastructure Manager to apply configuration changes automatically.
You should use a linear pipeline when stages have clear dependencies and must run one after another, as it runs each stage sequentially, while a parallel pipeline runs independent stages at the same time to speed up the overall process and a triggered pipeline starts automatically upon events like a code push or a scheduled time.
Cloud Build runs builds based on a configuration file that lives in the repository alongside the code, keeping the pipeline definition versioned with the software, and secrets like API keys and passwords are stored securely in Secret Manager and retrieved at runtime; access is controlled through IAM so only the pipeline service account can read them.
Artifact Registry stores container images with vulnerability scanning enabled, and the pipeline can verify image signatures before deployment to ensure only images built from approved source code are released, protecting against malicious code injected into dependencies or build processes.
StudyCerts
StudyCerts