Professional Cloud DevOps Engineer

Designing Pipelines

Pipeline Architecture

A CI/CD pipeline is a set of steps that automatically builds, tests, and deploys code every time a developer pushes changes. When designing a pipeline, the team decides what happens at each stage: code is pulled from the repository, compiled or packaged, tested, and then deployed to an environment. The architecture must match the type of workload—application code, infrastructure definitions, or machine learning models—because each has different requirements. For example, an ML pipeline needs stages for data validation and model training that a standard application pipeline does not require.

CI/CD Patterns

Different patterns serve different needs. A linear pipeline runs each stage one after another and works well when stages have clear dependencies. A parallel pipeline runs independent stages at the same time, which speeds up the overall process. A triggered pipeline starts automatically when events occur, such as a code push or a scheduled time. The team chooses a pattern based on how quickly they need feedback and how complex their release process is.

Implementing and Managing Pipelines

Build and Deployment Automation

Cloud Build is Google's service that runs builds in the cloud. It takes source code, runs the steps defined in a configuration file, and produces artifacts like containers or deployable packages. The configuration file lives in the repository alongside the code, so the pipeline definition stays versioned with the software it builds. For infrastructure, teams often use Terraform or Infrastructure Manager to apply configuration changes automatically. For machine learning workloads, Vertex AI Pipelines provides a way to orchestrate ML workflows using Kubeflow or Tekton components.

Testing Integration

Continuous testing means running tests automatically every time code changes. Unit tests run first to check individual functions, then integration tests verify that components work together, and finally end-to-end tests simulate real user scenarios. The pipeline stops and notifies the team if any test fails, preventing bad code from reaching production. Test results are stored and tracked so the team can see trends over time, such as whether the code is becoming more reliable or if specific areas are breaking frequently.

Managing Pipeline Configuration and Secrets

Configuration as Code

Pipeline configuration lives as code in the repository, which brings the same benefits as application code: version control, review processes, and history tracking. Changes to the pipeline go through code review before being applied, reducing the risk of accidental misconfiguration. The configuration defines which steps run, what environment variables are set, and which resources are used at each stage.

Secret Management

Secrets such as API keys, passwords, and tokens must never appear in plain text in configuration files. Secret Manager stores secrets securely, and the pipeline retrieves them at runtime when needed. Access to secrets is controlled through IAM, ensuring only the pipeline service account can read them. This separation means secrets are not baked into artifacts and can be rotated without changing the pipeline code.

Securing the Deployment Pipeline

Pipeline Security Controls

The pipeline itself must be protected because a compromised pipeline can deploy malicious code. Access to pipeline settings is restricted to authorized users through IAM roles. The service account that runs the pipeline should have only the permissions it needs, following the principle of least privilege. Audit logs track who made changes to the pipeline and when, supporting accountability and compliance requirements.

Supply Chain Security

Supply chain security focuses on ensuring that the software being deployed comes from trusted sources and has not been tampered with. Artifact Registry stores container images and other artifacts with vulnerability scanning enabled. The pipeline can verify image signatures before deployment, ensuring that only images built from approved source code are released. This protects against attacks that try to inject malicious code into dependencies or build processes.