Prepare and test your skills
Prepare and test your skills
Establishing recovery objectives means translating business tolerance for downtime and data loss into a specific disaster recovery architecture. The Recovery Time Objective (RTO) (opens in a new tab) is the time allowed before service runs again after a disaster, and the Recovery Point Objective (RPO) is the amount of data that can be lost. Smaller RTO and RPO values need more ready infrastructure and more frequent replication, so cost and operational complexity rise as the objectives approach zero.
Recovery Time Objectives and Recovery Point Objectives are measurable targets that control disaster recovery design. RTO controls how ready infrastructure, software, and data must be at the recovery site, while RPO is controlled by backup frequency and backup retention. Backup frequency sets the maximum gap between the last backup and a disaster, and retention sets how far back a recovery point can be moved to avoid logical errors.
Organizations group applications into criticality tiers so each tier gets its own RTO and RPO for zone and region outages. A Tier 1 payment or storefront system may require zero RTO and zero RPO for both zonal and regional outages, while a Tier 2 CRM system may allow 15 minutes for a zone outage and one hour for a region outage. Tier 3 back-office systems may allow one hour for a zone outage and 12 hours for a region outage. Together, tier plus RTO and RPO answer whether the application must span zones in one region or zones in multiple regions, and which Google Cloud products it can depend on.
Business-critical operations subject to RTO and RPO must depend only on data plane components that continuously process operations. The application must not depend on management plane operations that manage configuration state, such as a VM-creation API or an update of an IAM permission, because those controls may not be available during the outage window.
Scope of loss defines the boundary an architecture must survive: a single zone, a whole region, or multiple regions. Zonal resources, such as Compute Engine virtual machines and Persistent Disk, live in one zone and fail if that zone fails. Regional resources are composed across multiple zones in a region and automatically serve from another zone if one zone is interrupted. Multi-regional resources are distributed within and across regions and generally provide higher resilience than regional resources.
High availability (HA) and disaster recovery (DR) use different scopes and replication. HA typically applies to regional deployments with synchronous replication between zones, supporting zero or near-zero RTO and RPO in an active-active topology without a single point of failure. DR typically applies to multi-region deployments with asynchronous replication between regions, which lowers latency but creates risk of data loss and requires defined failover operations. For cross-region needs, managed multi-regional services such as Spanner (opens in a new tab) and Cloud Storage provide built-in redundancy and synchronization.
Exam tip: HA means synchronous replication across zones in one region for near-zero RTO and RPO, while DR means asynchronous replication across regions where failover steps and replication lag affect RTO and RPO.
Cold, warm, and hot standby patterns describe how much of the recovery environment runs before a disaster. In a cold pattern, backups and images are prepared but compute does not run until failover, so data flows from the primary to Cloud Storage (opens in a new tab) on a schedule and traffic flows from Cloud DNS to the primary until DNS is switched to the recovery site. In a warm pattern, a minimal system such as a small database server runs continuously and receives replicated transactions, so traffic still requires DNS failover but data loss is smaller. In a hot pattern, both environments actively serve traffic behind load balancing with continuous replication, so failover is fastest.
A three-column comparison of cold, warm, and hot standby showing what runs before failover, replication and traffic failover, and the cost versus RTO and RPO trade-off.
The three patterns trade cost against recovery speed in a fixed order:
| Pattern | What runs before failover | RTO and RPO | Choose when |
|---|---|---|---|
| Cold standby | Backups in Cloud Storage, custom images, and a Deployment Manager template; no running servers | Largest RTO and RPO, lowest cost | Recovery can tolerate hours of rebuild and restore from backups and logs |
| Warm standby | Minimal database server runs and receives asynchronous replication; app servers restored from snapshots | Small RTO and RPO, medium cost | Budget and availability must balance without full redundancy |
| Hot standby | Full secondary environment runs and serves traffic with synchronous or near-synchronous replication | Near-zero RTO and RPO, highest cost | External or real-time workloads cannot tolerate downtime or data loss |
A cold recovery to Google Cloud first copies backups and transaction logs to Cloud Storage, then creates the network, custom images, and deployment template in advance. During failover, the deployment template creates servers in Google Cloud, the latest backup and logs are applied to the database, the system is tested, and then Cloud DNS points users to the new web server. A warm recovery keeps connectivity through Cloud Interconnect and a VPN, keeps the standby database running to receive replication, uses Compute Engine snapshots for web and application binaries, and scales up the minimal instance to production size during failover. A hot design distributes traffic across regions with Cloud DNS and Cloud Load Balancing, uses regional load balancers or a global load balancer, stores data in multi-regional services, and overprovisions capacity across three zones per region to absorb failover.
Orchestrating traffic management and failover means designing automated traffic paths that keep service continuity during zonal or regional outages. In Google Cloud, Cloud Load Balancing (opens in a new tab) steers traffic to healthy backends based on health checks, while Cloud DNS redirects client requests between environments during failover and failback. Business-critical operations that depend on these paths must rely only on data plane components that continuously process operations, not on management plane operations such as VM-creation APIs or IAM permission updates.
Global load balancing provides automated failover across regions without manual intervention during a regional outage. Each global load balancer is distributed across all regions, so traffic flows from clients into the Google front end nearest the client and then to healthy backend services in available regions. If a region becomes unreachable, cross-regional load balancing automatically diverts traffic to backends in another region, which supports zero or near-zero RTO when backends and data are already replicated.
The resilience of the application depends not just on the load balancer scope but also on the redundancy of backend services. A global external Application Load Balancer is resilient to zone and region outages, but the application can continue to serve requests only if backends exist in more than one zone or region. Common backend designs use regional managed instance groups (MIGs) with identical Compute Engine virtual machines (VMs) in multiple zones, so the load balancer can forward requests to surviving VMs when health checks mark VMs in the failed zone as unhealthy.
Load balancer scope determines which outage the front end survives:
| Load balancer scope | Distribution | Outage resilience |
|---|---|---|
| Global | Distributed across all regions | Resilient to zone and region outages |
| Cross-region | Distributed across multiple regions | Resilient to zone and region outages |
| Regional | Distributed across multiple zones in the region | Resilient to zone outages; an outage in a given region affects the regional load balancers in that region |
Regional load balancing handles failover inside a single region when a zone experiences issues. Each regional load balancer is distributed across multiple zones in the region, and traffic flows from the regional forwarding rule to VMs in all zones within that region. When a health check determines that backends in one zone are unhealthy, the load balancer quickly routes traffic to VMs in another zone.
Zonal resources such as Compute Engine (opens in a new tab) VMs and Persistent Disk require this composition because a zone interruption can affect all resources in that zone. Strategies include running identical VM instances in multiple zones with instance templates and managed instance groups, using regional Persistent Disk to synchronously replicate data to another zone, and reserving external IP addresses in global or regional mode because the mode affects availability during a regional failure. For high availability, many managed services provide synchronous replication between zones by default, which enables an active-active topology with no single point of failure.
Cloud DNS redirection is a durable control for multi-region failover because Cloud DNS zones are global resources resilient to zone and region outages. Traffic distribution flows from DNS resolution to the selected load balancer endpoint: distribute user traffic across regions (opens in a new tab) by using Cloud DNS with a regional load balancer deployed in each region, or use a single global load balancer as the front end depending on requirements and criticality.
DNS redirection can be manual or automated, and the choice directly affects RTO. If the DNS service does not automatically reroute traffic on health-check failure, operators must manually reconfigure DNS routing to send traffic to the system that is still up. During recovery, operators configure Cloud DNS records to point to the web server on Google Cloud, for example by using an anycast IP address behind a load balancer with multiple web servers behind it, and then reverse the records to point back to the primary environment during failback.
The ordered failover and failback sequence defines the steps for moving production traffic to a recovery environment and then returning it to the primary. A typical cold-recovery failover moves traffic only after the recovery environment is tested:
Failback reverses the flow only after the primary can support production workloads again. Data flows from the active recovery database back to the primary database first, then traffic is blocked on the recovery side before DNS is switched:
Exam tip: A regional load balancer fails with its region, while a global load balancer survives a regional outage — but a configuration error in a single global load balancer can affect all geographies.
Defense-in-depth fallbacks protect a geographically distributed application against an outage or misconfiguration of its global front end. Global resources are resilient to physical failures, but their configuration is globally scoped, so a critical error can make the global load balancer a single point of failure across all geographies. Changes to global resources must therefore be managed carefully.
The fallback pattern keeps regional load balancers deployed as backups behind Cloud DNS. Clients prefer the global load balancer under normal conditions, but traffic flows from clients to the nearest regional load balancer if the global load balancer is unavailable. This automated failover or bypass strategy uses Cloud DNS and regional backup load balancers to maintain accessibility even when the global front end is affected.
Implementing data resiliency and replication means keeping data persistent and available during zone outages, regional disasters, and data corruption events. On Google Cloud, Cloud Storage (opens in a new tab) provides multi-regional and dual-regional storage classes for object redundancy, managed databases provide built-in synchronous replication, and backup features such as scheduled backups, point-in-time recovery (PITR), and versioning protect against deletion and corruption. Together these controls let an architecture meet RTO and RPO targets for business continuity.
Cloud Storage multi-regional and dual-regional buckets store each object in at least two geographically separate areas, regardless of storage class. This cross-region redundancy keeps data accessible during large-scale disruptions such as natural disasters and supports 99.99% availability. Data synchronization between the underlying regions occurs asynchronously, so a very recent write may not yet be present in the second area when a disruption begins.
Dual-regions achieve redundancy by using a pair of regions that you choose, while multi-regions achieve redundancy by using any combination of data centers in the specified multi-region, which might include data centers that are not explicitly listed as available regions. Single-region buckets keep data at rest in one region and do not provide geographic redundancy by themselves. Choose dual-regions when you need to control the exact region pair, and choose multi-regions when you want Google to manage placement across the wider geographic area.
| Bucket location type | How replica locations are selected | Redundancy behavior |
|---|---|---|
| Dual-regions | Pair of regions that you choose | Objects stored simultaneously in two geographically separate regions |
| Multi-regions | Any combination of data centers in the multi-region | Objects stored in at least two geographically separate areas for maximum availability |
| Two single-region buckets | Two buckets you create and manage | High confidence of cross-region write through dual-write or Cloud Storage copy from source bucket to destination bucket |
Stronger RPO controls reduce how much newly written data can be lost when replication is asynchronous. Turbo replication replicates 100% of newly written objects between both regions of a dual-region pair within 15 minutes, which gives a 15-minute RPO for that pair. Cross-bucket replication asynchronously replicates data from a source bucket to a destination bucket in a different location for custom topologies. When you need high confidence that an object has reached the alternative region to meet strict RTO and RPO values, write the object to two single-region buckets by dual-writing it or by writing to one bucket and then copying it to the second bucket.
If requirements restrict you to a single region, geographic redundancy cannot be achieved with Google Cloud alone. In that case, adopt a multi-cloud or hybrid strategy that stores copies in another cloud or on-premises system in the same geographic area, using tools such as Storage Transfer Service for on-premises data, interoperability with Amazon S3, or open-source object stores such as MinIO and Ceph. When locality rules permit encryption as a workaround, customer-managed encryption keys (CMEK) or customer-supplied encryption keys (CSEK) can allow the use of multi-region or dual-region buckets because data cannot be decrypted without the keys.
Exam tip: In BigQuery, a multi-region location does not provide cross-region replication nor regional redundancy — data is stored in a single region within that geographic location, so a separate backup or replica is still required for regional loss.
Firestore in Datastore mode (opens in a new tab) automatically handles replication for infrastructure outages without additional configuration or provisioning. A regional database synchronously replicates data across at least three zones and provides 99.99% availability. A multi-region database synchronously replicates data across five zones in three regions, with two serving regions and one witness region, and provides 99.999% availability for maximum durability.
Firestore in Datastore mode also protects against data disasters such as accidental deletion or modification through scheduled backups and PITR. Scheduled backups can run daily or weekly with up to 14 weeks of retention, and a database restores from a backup to a new Datastore mode database in the same project at lower cost than PITR recovery. PITR supports reads from up to seven days in the past at 1-minute granularity, with an RTO of 0 for surgical write-back and an RPO of 1 minute, which is better than backups for recovering only the data that changed.
For longer retention, PITR can create a full-database export to Cloud Storage that is kept indefinitely, although recovery from that export is generally more expensive than recovery from a backup. A database can also be cloned from a past point in time, from up to seven days in the past when PITR is enabled and from up to one hour in the past when PITR is not enabled.
BigQuery (opens in a new tab) stores data with high availability and durability by spreading storage and compute across multiple availability zones, but it does not automatically keep a backup or replica in another geographic region. A hard regional failure that destroys an entire region could cause loss of data stored in that region, while soft zonal or regional failures are expected to cause query failures or temporary unavailability without data loss. To survive regional loss, continuously replicate data to a geographically distinct region with cross-region dataset replication and provision slots in both locations, or use managed disaster recovery to control failover, and avoid backing up a multi-region such as US or EU to an overlapping region such as us-central1.
BigQuery also supports time travel for seven days to query and restore deleted, modified, or corrupted data, including deleted tables, and table snapshots that explicitly preserve data within the same region through a metadata-only operation with no additional storage bytes. Snapshots help with accidental deletion but do not increase durability against regional loss. For single-region designs, schedule exports with Cloud Scheduler or Managed Service for Apache Airflow to Cloud Storage in formats such as Avro with SNAPPY compression or JSON with GZIP compression, or convert to columnar Parquet or ORC for high compression and compatibility with Hive, Presto, and Spark.
Bigtable production instances require a minimum of two clusters in unique zones of a region, and replication between those clusters is automatically managed by Google. Bigtable synchronizes data between the clusters to create a separate, independent copy in each zone, which allows incoming traffic to fail over to another cluster in the same instance. Filestore Regional instances replicate files synchronously across three zones in a region for high availability against zone outages, and a Filestore instance can be replicated to a different region for robustness against region outages.
Cloud Storage data protection features prevent permanent loss from accidental or malicious deletion and support legal or regulatory retention. Object holds block deletion or overwrite of individual objects through metadata flags, soft delete retains recently deleted objects and buckets for a set period with a seven-day default, and Object Versioning preserves overwritten or deleted live objects as noncurrent versions that remain accessible until explicitly removed. Bucket Lock enforces a minimum retention duration for all objects in a bucket and locks the policy so it cannot be reduced or removed, while Object Retention Lock applies an immutable retain-until time to specific objects.
The five main controls each solve a different protection problem:
Versioning and soft delete enable rollback after accidental overwrites or ransomware events because a prior iteration or recently deleted object can be restored. Retention locks are different because they enforce immutability for compliance rather than simply keeping a recoverable copy. Choose holds, versioning, and soft delete for operational recovery, and choose Bucket Lock or Object Retention Lock when objects must be kept unchanged for a required period.
Backup and DR Service is a managed backup and disaster recovery service that provides secure storage and centralized backup management at scale for virtual machines (VMs) and other workloads running on Google Cloud. The service backs up Persistent Disk snapshots from Compute Engine VMs to Cloud Storage buckets, provides comprehensive monitoring, auditing, and reporting for compliance, and helps defend against ransomware through insights into security threat events. Data flows from the protected Compute Engine disks to the backup vault in Cloud Storage, and recovery flows back from that vault to new or restored workloads.
Storage Transfer Service migrates or replicates data to, from, or between object and file storage systems such as Cloud Storage, Amazon S3, and on-premises storage for backup and migration. Transfer Appliance moves very large amounts of offline data from on-premises to Cloud Storage using a physical appliance. For append-only warehouses partitioned by date, copy only the required recent partitions to a new table before running an export job, then transfer the files with the gcloud storage command, accounting for export limits and possible egress charges when data leaves Google Cloud.
Recovery Time Objective (RTO) is the time allowed before service runs again after a disaster, while Recovery Point Objective (RPO) is the amount of data that can be lost. RTO controls how ready infrastructure, software, and data must be at the recovery site, while RPO is controlled by backup frequency and backup retention. Smaller RTO and RPO values need more ready infrastructure and more frequent replication, so cost and operational complexity rise as the objectives approach zero.
Cold standby keeps backups in Cloud Storage, custom images, and a Deployment Manager template with no running servers for the largest Recovery Time Objective (RTO) and Recovery Point Objective (RPO) at the lowest cost, and you should choose it when recovery can tolerate hours of rebuild and restore from backups and logs. Warm standby runs a minimal database server with asynchronous replication for small RTO and RPO at medium cost, and you should choose it when budget and availability must balance without full redundancy. Hot standby runs a full secondary environment serving traffic with continuous replication for near-zero RTO and RPO at the highest cost, and you should choose it when external or real-time workloads cannot tolerate downtime or data loss.
High Availability (HA) typically applies to regional deployments with synchronous replication between zones, supporting zero or near-zero Recovery Time Objective (RTO) and Recovery Point Objective (RPO) in an active-active topology without a single point of failure. Disaster Recovery (DR) typically applies to multi-region deployments with asynchronous replication between regions, which lowers latency but creates risk of data loss and requires defined failover operations.
A global load balancer is distributed across all regions and is resilient to zone and region outages, while a regional load balancer is distributed across multiple zones in the region and is resilient to zone outages but fails with its region. An application behind a global external Application Load Balancer can continue to serve requests only if backends exist in more than one zone or region, such as regional managed instance groups (MIGs) with identical Compute Engine virtual machines (VMs) in multiple zones that receive traffic when health checks mark failed-zone virtual machines as unhealthy.
An enterprise runs a mission-critical transactional workload on a Cloud SQL instance in the us-central1 region. The organization needs to design a disaster recovery (DR) strategy that meets the following business continuity requirements:
us-east4) with minimal Recovery Time Objective (RTO).Which replication and recovery strategy should the cloud architect implement?
Last updated