ACID Compliance and Availability Design Choices

Design for availability and partition tolerance in distributed systems is the practice of keeping globally distributed data systems accepting reads and writes when zones, regions, or network links fail. On Google Cloud this is achieved by spreading work across failure domains, removing single points of failure, and choosing replication that keeps serving traffic during a partition while managing consistency and latency trade-offs.

1.1Multi-Region Topology for Partition Survival

Multi-region topology for partition survival uses at least two regions with three zones in each region so loss of one zone or region leaves healthy capacity. Cloud Spanner (opens in a new tab) supplies built-in redundancy and synchronization across regions for relational data, while Cloud Storage multi-region or dual-region buckets store data redundantly in at least two geographic locations. Traffic flows from users to Cloud DNS and then to a regional load balancer in each region or to a global load balancer, so requests still reach the surviving region when the other region is partitioned away.

Prefer resources with wider locational scope to avoid rebuilding from backups. Use regional resources instead of zonal resources, and use multi-regional or global resources instead of regional resources where possible. In each region, provision capacity in three zones rather than two and overprovision by a third more than the estimate, so remaining zones can absorb the surge when one zone fails.

1.2Eliminating Single Points of Failure

Eliminating single points of failure means no single server, database, disk, or zone can stop the entire stack when a partition isolates it. Redundant resources sit in different zones and regions, and traffic is balanced evenly across them so failure of one failure domain leaves adequate capacity in every tier. Synchronous replication within a region keeps both copies identical before acknowledging a write, which preserves consistency during a zonal partition at the cost of write latency between zones.

Common controls for removing single points of failure include:

• Deploy more than one application server and more than one database across zones.
• Use Compute Engine (opens in a new tab) managed instance groups to automatically recreate failed virtual machines, with regional groups and regional Google Kubernetes Engine clusters surviving zone outages.
• Distribute requests with Cloud Load Balancing (opens in a new tab) and health checks that stop sending traffic to unresponsive virtual machines.
• Use high availability configurations for databases such as Cloud SQL, including cross-zone read replicas for near active-active behavior.
• Use regional persistent disks or Hyperdisk Balanced High Availability, which synchronously replicate data between two zones in the same region.

1.3Aggregate Availability and Service Dependencies

Aggregate availability is the end-to-end availability of a multi-tier stack calculated from the availability of each dependent tier. For tiers that must all succeed in sequence, called chained services, aggregate availability is the product of each tier: tier1_availability x tier2_availability x tierN_availability. Four tiers at 99.9% each yield only 99.6% aggregate availability, and aggregate availability is always lower than the least-available tier.

Where feasible, choose parallelization over chaining to raise availability during partial failures. Two services at 99% each yield 98% when chained, but 99.99% when parallelized because each service runs independently and survives failure of the other, calculated as 1 - (1 - A) x (1 - B). Reducing stack depth, selecting services with stronger uptime service level agreements, and weighing extra nines against operational complexity, latency, and cost all change the aggregate result. For scale, 99.9% allows about 86 seconds of downtime in 24 hours, while 99% allows about 864 seconds, or ten times more.

1.4High Availability Versus Disaster Recovery

High availability and disaster recovery solve different partition scopes with different replication trade-offs. High availability covers zones within one region with synchronous replication and active-active topology, while disaster recovery covers two or more regions and typically requires self-managed asynchronous replication with manual or semi-automated failover. Recovery time objective answers how long after a disaster until service resumes, and recovery point objective answers how much data can be lost.

The table shows why business-critical operations that must meet strict recovery objectives must depend only on data-plane components that continuously process operations, not on management-plane operations such as virtual machine-creation APIs or Identity and Access Management permission updates.

1.5Managed Multi-Regional Services and Consistency Trade-Offs

Managed multi-regional services let architects prioritize availability and partition tolerance without building custom cross-region failover. Cloud Spanner provides distributed, synchronized redundancy across regions for relational data that still needs strong consistency, while Cloud Storage dual-region and multi-region buckets write metadata synchronously across regions and replicate object data asynchronously. Cloud SQL high availability with cross-zone replicas provides an RPO close to 0 between regions in a near active-active topology, and Filestore regional instances synchronously replicate shared file data across three zones in a region.

The core trade-off is synchronous versus asynchronous replication across a partition. Synchronous replication preserves consistency and minimizes data loss but adds latency and cost and can reduce write availability if replicas cannot acknowledge. Asynchronous replication keeps writes available and fast during a partition but accepts eventual consistency and potential loss, as with Cloud Storage dual-region turbo replication with an RPO of 15 minutes. For zonal or regional resources without built-in multi-region support, replicate data to a secondary region, to Cloud Storage, or to snapshots stored in multi-region locations, and test failover regularly.

Implementing transactional integrity in event-driven and stream processing architectures means each event is processed atomically, consistently, and durably even when zones, regions, or individual tasks fail. On Google Cloud this outcome is built by combining durable ingestion with regional execution that can retry, replicated state that preserves commits, and active-active topologies that keep the data plane running.

2.1Durable Ingestion with Pub/Sub and Cloud Storage

Durable ingestion is the foundation that prevents accepted events from being lost before processing completes. Pub/Sub (opens in a new tab) encrypts all messages at rest and in transit by default with Google-owned and Google-managed keys and supports customer-managed encryption keys for application-layer encryption. Message data can be pinned to specific locations with message storage policies, so residency rules are enforced while durability is preserved.

Data flows from producers to Pub/Sub topics, then to subscriptions and processing tasks, and finally to durable sinks such as Cloud Storage (opens in a new tab) buckets. Data in regional buckets is replicated synchronously across multiple zones in the region, while data in dual-region or multi-region buckets is stored redundantly in at least two locations with metadata written synchronously and object data replicated asynchronously. For dual-region buckets, turbo replication provides a recovery point objective of 15 minutes. Choose regional buckets when synchronous zonal durability is required, and choose dual-region or multi-region buckets when survival of a regional outage matters more than synchronous cross-region commits.

2.2Reliable Execution with Regional Compute and Retries

Reliable execution with regional compute and retries ensures that a failed task does not break atomic processing of the stream. Cloud Run is a regional service that stores data synchronously across multiple zones in a region and automatically load-balances traffic across those zones. If a zone outage occurs, Cloud Run jobs continue to run without data loss, but if a region outage occurs, the jobs stop until Google resolves the outage.

This retry-and-checkpoint pattern separates processing from storage: processing may restart in another zone, while the stored source data and checkpoints remain available in the selected location. Business-critical operations subject to recovery objectives must depend only on data plane components that continuously process operations, not on management plane operations such as a virtual machine-creation API or an Identity and Access Management permission update. Design stream processors so redelivery from Pub/Sub plus a checkpointed sink produces the same final state after a retry.

2.3Consistent and Available State with Replicated Databases and Disks

Consistent and available state with replicated databases and disks is where atomicity, consistency, isolation, and durability guarantees meet replication trade-offs. Spanner is a distributed, multi-regional database that provides built-in redundancy and synchronization across regions and supports active-active deployments with minimal manual recovery. Cloud SQL with its high availability feature provides a near active-active topology with read replicas across zones and an RPO between regions that is close to 0.

The choice of state determines the consistency, latency, and cost trade-off:

The table shows the central rule: use synchronous options when every commit must be durable before acknowledgment, and accept asynchronous cross-region replication when lower latency and lower cost matter more than zero data loss. Regional persistent disks and Filestore regional instances also replicate data synchronously across zones, with Filestore regional instances replicating across three zones in a region.

2.4Aggregate Availability in Chained Pipeline Tiers

Aggregate availability in chained pipeline tiers measures how the reliability of each pipeline tier multiplies into end-to-end uptime. A multi-tier stack calculates aggregate availability as tier1_availability x tier2_availability x tierN_availability, so the total is always lower than the least-available tier. Four services at 99.9% each yield 99.6% aggregate availability, and two chained services at 99% each yield only 98%. Parallelized services use 1 - (1 - A) x (1 - B), so two 99% services in parallel yield 99.99% because each runs independently.

This math directly affects event-driven design because ingestion, processing, and state form a chain where any tier can break atomic completion. Where feasible, choose parallelization over chaining so one branch can continue when another is impaired. Increasing nines costs more in operational complexity, latency, and spend: 99.9% allows about 86 seconds of downtime per day, while 99% allows 864 seconds. Select services with uptime service-level agreements that exceed the pipeline target so cloud downtime plus application downtime still meets the business requirement.

Eliminate single points of failure so one failed component does not force duplicate or lost transactions:

• Do not deploy a single application server or database for any tier that must commit exactly once.
• Recreate failed virtual machines automatically with managed instance groups, using regional groups to survive zone outages.
• Distribute traffic evenly with load balancing, using Cloud DNS plus a regional load balancer in each region or a global load balancer based on criticality.
• Use high availability configurations for databases such as Cloud SQL.
• Improve data availability with regional persistent disks that use synchronous replication between zones.

2.5Recovery Objectives and Active-Active Design for Streams

Recovery objectives and active-active design for streams translate atomicity and availability goals into recovery time and recovery point objectives by outage scope. High availability generally applies to regional deployments with synchronous replication between zones and zero or near-zero RTO and RPO. Disaster recovery (DR) (opens in a new tab) generally applies to multi-region deployments, where replication without multi-regional services is typically asynchronous and requires manual or semi-automated failover with larger RTO and RPO.

For critical pipelines, use a multi-region topology across at least two regions with three zones in each region, because failure in one location can overload a second location. Prefer wider scopes first by using regional instead of zonal resources and multi-regional or global resources instead of regional resources, which avoids restoring from backups. In each region, leverage three zones and overprovision capacity by a third more than the estimate so surviving zones can absorb failover traffic. Distribute user and event traffic across regions, store pipeline state in Spanner and Cloud Storage multi-region services, store persistent disk snapshots in multi-region locations, replicate regional resources to a secondary region, and test the failover plan regularly.

Analyzing GCP database offerings against ACID properties means comparing how each managed service replicates data, handles failure, and preserves availability when strict data integrity is required. Cloud Spanner, Cloud SQL, Firestore, and Bigtable differ in replication scope, copy behavior, and failover mechanics, which directly affects durability and availability for transactional workloads.

3.1Cloud Spanner

Cloud Spanner (opens in a new tab) is a distributed database that provides built-in redundancy and synchronization across zones and regions. In any regional configuration, Spanner maintains three read-write replicas, each in a different Google Cloud zone in that region. Each read-write replica contains a full copy of the operational database that is able to serve read-write and read-only requests, so data flows from the write path to all three zonal copies and reads can be served from any of them.

Cloud Spanner multi-region deployments extend this model across regions with two read-write regions and one witness region containing a witness replica. The multi-region deployment provides a consistent environment across multiple regions without manual replication steps, which supports active-active topologies with minimal recovery work. When locality restrictions apply, the locations of all regions in the configuration must be validated against residency requirements.

3.2Cloud SQL

Cloud SQL (opens in a new tab) is a managed relational service that uses a primary and standby model for high availability within a region. A Cloud SQL instance configured for high availability is also called a regional instance and is located in a primary and secondary zone in the configured region. If the primary database fails, Cloud SQL fails over automatically to the standby database without requiring a change to the database endpoint, but architects must understand the typical failover time from primary to standby.

Cloud SQL supports cross-region replicas for protection against regional outages, with a recovery point objective between regions that is close to 0 in near active-active topologies. If a disaster occurs, the read replica in a different region can be promoted to become the new primary. Read replicas can be configured for high availability in advance so they do not need additional changes after promotion, and read replicas can have their own cross-region cascading replicas that offer immediate protection after replica promotion.

3.3Firestore

Firestore (opens in a new tab) is a managed document database that offers a choice between regional and multi-region storage locations with different replication guarantees. Data in a regional location operates in a multi-zone replicated configuration, so the data survives a zone loss within that region. Data in a multi-region location operates in a multi-zone and multi-region replicated configuration, where replication spans zones and regions.

Firestore multi-region locations can withstand loss of entire regions and maintain availability without data loss, which provides the highest durability and availability described for Firestore. A multi-region location should be selected when locality restriction requirements permit it and the goal is to maximize availability and durability. A regional location trades that cross-region resilience for tighter control over where data is stored.

3.4Bigtable

Bigtable is a managed wide-column service deployed as zonal clusters within a region. Bigtable is available in zonal locations in a region, and production instances have a minimum of two clusters, which must be in unique zones in the region. Replication between clusters in a Bigtable instance is automatically managed by Google, so the application does not manage its own copy process.

Bigtable synchronizes data between the clusters, creating a separate, independent copy of the data in each zone where the instance has a cluster. Incoming traffic can fail over to another cluster in the same instance when one zone experiences issues, so the dependency is between clusters in the same instance rather than on an external backup. This design protects against zonal failure inside a region but does not by itself provide cross-region resilience.

3.5Comparing Offerings for Transactional Workloads

Comparing offerings for transactional workloads requiring strict data integrity means matching replication scope, copy completeness, and failover behavior to recovery time and recovery point requirements. High availability generally relies on synchronous replication between zones within a single region with zero or near-zero RTO and RPO, while disaster recovery across regions without multi-regional services typically relies on asynchronous replication with manual or semi-automated failover. Managed multi-regional services reduce manual recovery steps compared with composing failover between separate regional resources.

Key Facts

• Cloud Spanner maintains three read-write replicas in three different zones in a regional configuration, and multi-region configurations use two read-write regions plus a witness region.
• High availability uses synchronous replication within a region with zero or near-zero RTO and RPO, while disaster recovery across regions typically uses asynchronous replication with higher RTO and nonzero RPO.
• Aggregate availability for chained tiers is the product of each tier, so four tiers at 99.9% yield 99.6%, while two parallel 99% services yield 99.99%.
• Cloud Storage dual-region turbo replication writes metadata synchronously and replicates object data asynchronously with an RPO of 15 minutes.
• Firestore multi-region withstands loss of entire regions without data loss, while Firestore regional survives only zone loss within one region.