Prepare and test your skills
Prepare and test your skills
Site Reliability Engineering (SRE) (opens in a new tab) is Google's approach to balancing fast delivery of new features with availability, latency, performance, and capacity. SRE starts from the observation that outages often follow new code and environment changes. It therefore uses structured testing, gradual deployment, incident response, and learning from failure to allow necessary change without causing unnecessary downtime. Organizational transformation extends the same idea to people and culture by connecting development, operations, and related teams through shared processes and tooling.
DevOps (opens in a new tab) is a set of culture, processes, and tooling that breaks down silos between development, operations, and related teams. Development teams focus on feature velocity while operations teams focus on stability, and DevOps connects them through shared pipelines, automation, runbooks, and reviews. Because both sides use the same deployment pipeline and incident practices, quality and reliability improve without slowing releases. This shared responsibility is especially important during large-scale cloud migration, when cross-functional coordination determines whether operational changes succeed.
Design for testability means building an application so automated unit, integration, and system tests can verify behavior before code moves to the next deployment stage. Unit tests carry most of the testing load because they are quick to run and easy to maintain, while automated integration and system tests check how larger parts work together. Infrastructure as code makes higher-level testing easier because dedicated test environments can be created on demand with tools such as Terraform and torn down after tests finish. As test coverage grows, each code change creates less uncertainty, so teams can make more changes before reliability drops.
Continuous integration builds on this test foundation by running a robust test set on every commit to give fast feedback. Source commits flow into an automated test suite, and the suite's results decide whether the change can progress toward deployment. Cloud Build (opens in a new tab) and Jenkins support continuous integration on Google Cloud, while Cloud Deploy helps manage delivery pipelines. Incorporating unit, integration, and end-to-end tests into the pipeline validates changes early and reduces errors and regressions.
Automated deployment is the step that follows continuous integration once testing gives confidence that software is stable. The level of automation depends on organizational maturity, but the decision rule stays the same: choose a strategy that exposes a new version to larger audiences gradually while verifying behavior at each step. The strategy must also define clear provisions for rollback if problems occur. This gradual exposure trades a small delay in full rollout for a much smaller blast radius from a defective release, failed virtual machine, or zonal outage.
Adopting SRE practices for dealing with failure means accepting that some component failure is normal in large distributed applications. Careful design helps an application survive a defective release, unexpected termination of virtual machines, or an infrastructure outage that affects an entire zone, but unexpected events will still need human intervention. Structured incident management reduces impact and resolution time, and careful study of causes and responses protects the application against similar events in the future. Strong incident management and blameless postmortems, which focus on systems rather than blaming people, are key tenets of SRE.
Exam tip: Full Google SRE implementation is not required to gain benefit; adopting even a minimum set of incident management and blameless postmortem guidelines improves resilience.
Clear incident response procedures define who acts, how information moves, and what steps to follow during an incident. The procedures link the incident commander to investigators, communicators, and technical experts through coordinated roles, communication paths, and documented playbooks:
Retrospectives extend this learning after milestones, major incidents, or testing cycles. Teams reflect on what went well, identify areas for improvement, and use a structured Start-Stop-Continue model to produce actionable steps, including where change-management automation can further improve reliability.
Validating resilience means periodically testing how architecture behaves under failure and load because users, traffic, business priorities, and dependent services keep changing. The guiding idea is that the best way to avoid failure is to introduce failure in a controlled way and learn from it. Simulation must verify application behavior and confirm that expected alerts and metrics are generated. A structured escalation validates behavior at each stage in order:
A service mesh such as Istio can inject faults at the application layer instead of killing pods or machines, including delays that simulate latency or overload and aborts that mimic upstream failures. Scaling validation uses automated nonfunctional, performance, and load testing to verify that key metrics stay within expected levels under varying load, that resource creation stays within expected range, and that edge cases such as maximum scaling limits or scale-down followed by sudden load increase behave as designed.
Optimizing teams means closing skill gaps and improving structure and culture so environment improvements can continue. Teams design and implement a training program, often using professional Google Cloud training and certifications, and they reshape team structure and culture for their own history and constraints. Transformational leadership provides a framework for executing and measuring organizational change toward DevOps, while SRE guides show how to put SRE principles into practice with concrete examples.
Continuous improvement depends on eliminating toil through automation and monitoring everything that matters. Per SRE guidance, toil is reduced by increasing automation, from short scheduled scripts to managed services such as Cloud Scheduler, Cloud Run, and Managed Service for Apache Airflow. A comprehensive monitoring system tracks essential metrics, at minimum the four golden signals, using Cloud Monitoring (opens in a new tab) and Cloud Logging, with longer analysis in BigQuery and optimization guidance from Recommender and Security Command Center. A culture of learning ties this together by treating failures as growth opportunities, encouraging experimentation and knowledge sharing, and creating psychological safety for cloud-native practices.
Deployment Strategies and Risk Mitigation is the practice of releasing changes in small, validated steps so downtime is minimized and a failed change can be reversed quickly. In Google Cloud, Cloud Build (opens in a new tab) builds and tests code, Artifact Registry stores versioned artifacts, and Cloud Deploy promotes those artifacts through target environments in a defined sequence. Automated tests and security checks act as validation gates between each stage, so only verified artifacts reach production.
Automated testing in continuous integration and continuous delivery (CI/CD) pipelines validates every change before deployment to reduce errors and regressions. Source code flows from Cloud Storage, Cloud Source Repositories, GitHub, or Bitbucket into Cloud Build, which executes the build to specification and produces artifacts such as Docker containers or Java archives. Running a robust set of tests on each commit gives fast feedback, so teams gain confidence that the software is stable enough to deploy.
Test coverage is layered to catch different kinds of failures before they reach users:
Higher test coverage reduces uncertainty from each code change, which means more changes can be made before reliability falls below an acceptable level. Testability improves further with infrastructure as code, because dedicated test environments can be created on demand and torn down when tests complete. Performance and load testing adds a nonfunctional gate: load testing simulates realistic traffic patterns, while stress testing pushes the system to its limits to find bottlenecks, often using Cloud Load Balancing (opens in a new tab) and load-testing services to generate traffic. Based on results, teams adjust resource allocation or tune application configuration for performance and scalability.
A deployment strategy controls how much exposure a new version gets and how the team reverses course if behavior is wrong. The core principle is to gradually increase exposure to larger audiences while verifying behavior along the way, with clear provisions for rollback if problems occur. To lower the risk of unanticipated issues, migration and release plans minimize the number and size of changes in each rollout, resolve issues in the first small rollout, and then repeat similar changes at larger scales.
Rollback readiness depends on keeping prior known-good versions available and on rehearsing the reversal path. A practical refactoring sequence for containerized workloads preserves rollback options throughout migration:
Each migration step needs its own tested rollback strategy and a maximum-allowed execution time that triggers rollback when exceeded. Teams must also assess which steps are potentially disruptive even with rollback, such as steps that could cause data loss, and prepare a preplanned manual procedure for every automated step in case automation itself fails. For connectivity changes, backup access paths prevent lockout, for example keeping public internet access as a backup while provisioning Partner Interconnect.
Exam tip: Keep artifacts in both the source repository and Artifact Registry during migration to enable rollback, and switch to Artifact Registry only after the Google Cloud environment is validated.
A secure automated release pipeline ensures that only verified, approved artifacts move from development to test to production. Cloud Deploy automates delivery to a series of target environments in a defined promotion sequence, while policies per environment require that only verified artifacts are deployed. Automation removes manual errors, creates standardized feedback loops, and enables efficient iteration, while also making it faster to detect and fix security bugs, narrow security reviews, and roll back to a known and safe version.
A left-to-right process flow from Source to Cloud Build to Artifact Registry to Cloud Deploy through a Binary Authorization gate to GKE and Cloud Run, with test, scan, and policy validation gates.
Security gates are embedded directly in the pipeline rather than added at the end. Artifact Analysis (opens in a new tab) automatically scans container images when they are uploaded to Artifact Registry, extracts system package information, and then continuously monitors scanned metadata against new vulnerability sources to create or remove findings. Web Security Scanner crawls App Engine, Compute Engine, and Google Kubernetes Engine (GKE) web applications to exercise inputs and detect cross-site scripting, code injection, mixed content, and insecure libraries. The Supply-chain Levels for Software Artifacts (SLSA) framework organizes these controls across source, build, and provenance.
Binary Authorization enforces the final approval before deployment to GKE or Cloud Run by checking configurable signatures called attestations on container images. At deployment time, Binary Authorization verifies that a trusted build system created the image, that the image complies with vulnerability signing policy, and that the image meets promotion criteria such as moving from development to quality assurance. If an attacker compromises the pipeline, this attestation check prevents untrusted code from running on the target platform.
The three core release services divide the flow by responsibility. The following table compares their roles in moving code from source to production:
| Service | Role in the release flow | What it controls |
|---|---|---|
| Cloud Build | Builds from source | Build execution and test validation before artifacts are produced |
| Artifact Registry | Stores versioned artifacts | Artifact lifecycle, access control, vulnerability scanning, and history for rollback |
| Cloud Deploy | Delivers to environments | Ordered promotion across development, test, and production targets |
Together, gradual exposure, continuous validation, and attested promotion keep change small, observable, and reversible. Small rollouts limit blast radius, automated tests and scans block bad changes early, and versioned artifacts plus tested rollback procedures restore service quickly when a release fails.
Governance, Compliance, and Change Auditing is the set of controls that keeps infrastructure changes safe, traceable, and reviewable in Google Cloud. Identity and Access Management (IAM) (opens in a new tab) roles and organization policies prevent risky actions before they happen, while Cloud Audit Logs and monitoring record what changed, who changed it, and whether services stayed available. Together they enforce separation of duties, minimize disruption, and provide evidence for internal standards and external regulations.
IAM governance limits who can act on sensitive resources so no single user holds conflicting power. Separation of duties is enforced for service account administration by not assigning Service Account User or Service Account Token Creator roles broadly at the project level, and for encryption by separating Cloud Key Management Service (Cloud KMS) key use from key administration. Least privilege extends this idea by removing admin privileges from service accounts, blocking automatic IAM grants to default service accounts, and restricting API keys to only needed hosts, apps, and APIs with rotation every 90 days or less.
Organization Policy provides centralized, hierarchical guardrails that constrain what can be configured in folders and projects. Organization Policy constraints can block default network creation, disable external IPv6 or VM serial-port logging to Stackdriver, restrict Cloud Run ingress and egress settings, and restrict authorized networks on Cloud SQL instances. Because the policy control plane manages these rules above individual projects, teams can adopt cloud-native change safely without relying only on manual review to catch misconfiguration.
Progressive deployment is the practice of rolling infrastructure changes to a small blast radius first, verifying behavior, then expanding. A change to VM machine types or Cloud Storage bucket classes moves through an ordered sequence:
This order matters because early detection limits impact, and only verified changes proceed to full rollout.
Control of global resources addresses changes to Virtual Private Cloud (VPC) networks and global load balancers that affect all zones and regions at once. Global resources are resilient to zonal and regional outages, so architectures often use a single VPC or global load balancer, which then becomes a single point of failure. A misconfigured forwarding rule can stop the frontend from receiving user requests even while backends remain healthy. To prevent this, the change-review process classifies any modification to global resources as a high-risk change that additional reviewers must verify and approve before deployment to production.
Exam tip: Treat a change to a global load balancer forwarding rule or VPC network as high-risk requiring extra approval, while a zonal VM change can roll out progressively.
Cloud Audit Logs answer who did what, where, and when for Google Cloud resources and form the auditable trail for every infrastructure change. Administrator Activity audit logs record configuration writes, Data Access audit logs record data reads and writes when enabled, and Access Transparency logs record Google administrator access. Logs flow from Cloud Logging to sinks for long-term storage, with retention policies locked by Bucket Lock so evidence cannot be altered or deleted early to meet compliance needs.
Change auditing depends on log metric filters and alerts that fire when governance itself changes:
When a filter matches, traffic flows from the log entry to Cloud Monitoring for alerting, so the security team learns about the change even if the actor had privileged access.
Monitoring of Google Cloud infrastructure availability uses the Google Cloud Service Health Dashboard (opens in a new tab) for global status and history of incidents, including duration, affected zones and regions, affected services, and workarounds. Personalized Service Health narrows that view to incidents relevant to a specific project or organization, with API access and configurable alerts plus RSS feeds for programmatic updates. Even with no reported outage, an application can still be unavailable from application errors or deleted configuration, so audit logs, asset inventory, and service health together prove whether disruption came from Google Cloud or from a tracked customer change.
Cloud Build builds code from source and validates it with tests, Artifact Registry stores the resulting versioned artifacts, and Cloud Deploy promotes only verified artifacts through development, test, and production in order. Automated tests and security checks act as validation gates between each stage, while versioned artifacts and tested rollback procedures allow quick reversal when a release fails.
Binary Authorization enforces final approval before deployment by verifying required attestations on container images. It checks that a trusted build system created the image, that the image complies with vulnerability signing policy, and that it meets promotion criteria such as moving from development to quality assurance, so compromised pipeline code cannot run on the target platform.
Changes to global Virtual Private Cloud (VPC) networks and global load balancer forwarding rules are classified as high-risk changes that additional reviewers must verify and approve before deployment to production. A misconfigured forwarding rule can stop the frontend from receiving user requests even while backends remain healthy, while zonal virtual machine changes can instead roll out progressively from a pilot in one zone to full deployment.
An enterprise is migrating its core transactional platform to Google Cloud. To meet aggressive release deadlines, the development team has increased deployment frequency, but consecutive unstable releases have caused major service disruptions and completely exhausted the service's monthly error budget.
The operations team wants to halt all cloud migrations and institute mandatory, multi-stakeholder manual change approval gates for every release, while the development team insists on maintaining release velocity.
Which approach aligns with Site Reliability Engineering (SRE) change management principles to resolve this conflict and balance innovation velocity with system stability?
Last updated