Design, migrate, and run scalable, highly available databases on Google Cloud across relational and NoSQL services, and track your readiness for the Professional Cloud Database Engineer exam.
Prepare and test your skills
Prepare and test your skills
Intra-region high availability and read scaling in AlloyDB Omni (opens in a new tab) keeps one regional cluster available through failure and able to serve growing read traffic. The design uses three building blocks inside the cluster: a primary instance that accepts writes, standby replicas that track the primary, and read pool instances that add horizontal read capacity. The AlloyDB Omni operator manages the relationship between them, so data flows from the primary instance to each standby replica and read consumer without manual copying.
A primary instance in AlloyDB Omni is the writable database instance that owns the authoritative copy of the data for the cluster. Application writes flow to the primary instance first, and replication then carries those changes from the primary instance to standby replicas and other readers. In container deployments, the primary instance runs inside a three-node Patroni cluster with a three-node etcd cluster as the configuration store and HAProxy in front of the cluster. The cluster depends on the operator to coordinate the primary instance and its replicas, so clients connect through the front-end routing layer while Patroni and etcd track which node is primary.
A standby replica in AlloyDB Omni is a full copy of the primary instance that stays current by continuously applying all data changes from the primary instance. The standby replica provides the failover target when the primary instance fails, the switchover target for planned moves, and it can also serve as a read-only instance for read traffic. Enabling this protection requires capacity for two parallel database instances, including storage for two complete copies of the data and compute resources for two instances running at the same time. High availability is enabled in the database cluster manifest under the cluster spec with the availability.numberOfStandbys parameter, which tells the operator how many standby replicas to create and keep synchronized.
The standby lifecycle in AlloyDB Omni covers automatic failover to the standby replica, manual failover for testing, switchover for planned transitions, and automatic healing when a standby replica becomes unhealthy. Failover moves write ownership from the failed primary instance to the standby replica, while healing rebuilds the standby replica from the current primary instance so the pair returns to a synchronized state. The standby replica therefore depends on the primary instance for its data, and availability depends on both copies having adequate storage, compute, and replication health. A standby replica serves reads only while it is synchronized and healthy, and it accepts writes only after it has been promoted to primary.
A read pool instance in AlloyDB Omni is a set of nodes that adds dedicated computing capacity for read-intensive workloads within the same cluster. The number of nodes in a read pool instance determines the overall computing capacity of that instance, so adding nodes or adding additional read pool instances scales reads horizontally without changing the primary instance. A cluster supports a maximum of 20 nodes across all of its read pool instances, which sets the upper bound for read-scale planning in one region.
Read pool capacity does not move automatically when a new cluster is created from an existing cluster. When a secondary cluster (opens in a new tab) is created, AlloyDB Omni copies and applies some primary cluster configuration, including point-in-time recovery (PITR) and backup configuration, but it does not create any read pool instances on the secondary cluster. Later changes to the primary cluster configuration are also not applied to the secondary cluster automatically and must be updated manually.
| Cluster setting | Copied to secondary cluster automatically | Required action |
|---|---|---|
| Point-in-time recovery (PITR) configuration | Yes | No manual copy needed at creation |
| Backup configuration | Yes | No manual copy needed at creation |
| Read pool instances | No | Add read pool instances manually on the secondary cluster |
A read pool instance is added to a secondary cluster through an ordered workflow:
Exam tip: A new secondary cluster inherits PITR and backup configuration but starts with zero read pool instances, even if the primary cluster has read pools.
Cross-region replication in AlloyDB is the use of a secondary cluster (opens in a new tab) in a different region or data center that continuously receives changes from a primary cluster (opens in a new tab) through asynchronous replication. The secondary cluster serves read traffic locally and stands ready as a disaster recovery target if the primary region fails. Because replication is asynchronous, disaster recovery planning must account for replication lag when defining Recovery Point Objective (RPO) and Recovery Time Objective (RTO).
A secondary cluster is a separate AlloyDB cluster linked to a primary cluster in another region. When the secondary cluster is created, AlloyDB copies some primary cluster configuration onto it, including point-in-time recovery (PITR) and backup configuration. The secondary cluster starts with one secondary instance and no read pool instances, no matter how many read pools exist on the primary cluster.
Configuration does not stay automatically in sync after creation. If the primary cluster configuration is updated later, those changes are not applied to the secondary cluster, and an administrator must manually update the secondary cluster to match. The secondary cluster must be created in a region different from the primary cluster, and its secondary instance ID must be unique within that cluster. Read pool instances can be added to the secondary cluster separately to scale local reads and to prepare capacity for failover.
To work with cross-region replication you need one of these Identity and Access Management roles in the Google Cloud project:
roles/alloydb.admin, the AlloyDB Admin predefined roleroles/owner, the Owner basic roleroles/editor, the Editor basic roleAsynchronous replication moves Write-Ahead Logging (WAL) data from the primary cluster to the secondary cluster by streaming, without waiting for the secondary cluster to confirm each write before the primary cluster commits. Data therefore flows from the primary database server to the secondary database server in order, and the secondary cluster normally lags slightly behind the primary cluster. That lag determines RPO, because any transactions not yet streamed when a regional failure occurs can be lost, and it affects RTO, because a lagging secondary cluster needs more time to catch up before promotion.
For AlloyDB Omni cross-data-center replication, the replication dependency requires external connectivity and sufficient WAL retention on the primary cluster. The primary database cluster manifest must set allowExternalIncomingTraffic: true so traffic flows from the secondary data center Kubernetes cluster to the primary data center Kubernetes cluster. When the primary cluster has high availability (HA) enabled, both the primary and standby database servers in the primary cluster depend on adequate WAL space to keep WAL segments until the remote secondary cluster fetches them.
Size WAL retention to support workload requirements, based on WAL generation rate, replication network characteristics, and replication lag between data centers:
| Parameter | Purpose | Default and rule |
|---|---|---|
wal_keep_size | Minimum size of past WAL files kept in the WAL directory for a lagging secondary server to fetch | Default 100MB; set high enough for peak write rate and network delay |
max_wal_size | Maximum size to let WAL grow during automatic checkpoints | Default 1504MB; must be set higher than wal_keep_size |
If a secondary server falls behind by more than wal_keep_size, the primary server may remove a WAL segment that the secondary server still needs. When that happens, the replication connection from the primary server to the secondary server is terminated, and downstream connections will also eventually fail.
Creating a secondary cluster establishes the replication relationship that disaster recovery depends on. For Kubernetes-based AlloyDB Omni, first ensure the operator and primary cluster exist, then enable replication in order:
allowExternalIncomingTraffic: true on the primary database cluster and configure wal_keep_size and max_wal_size on the primary and standby servers to handle cross-data-center lag.In hosted AlloyDB, creation is done from the primary cluster Overview page with Create secondary cluster or with the gcloud alloydb clusters create-secondary command that references the full primary cluster path. The request defines the secondary cluster ID, the secondary region, the encryption method, and the secondary instance ID. After creation, verify the secondary cluster status, update database flags or machine type on the secondary instance as needed, and add read pools for local performance.
Exam tip: If the primary instance is encrypted with a customer-managed encryption key (CMEK), the secondary cluster must also be encrypted with a CMEK key; Google-managed encryption cannot be mixed in that direction.
A disaster recovery protocol uses the secondary cluster as the recovery target that satisfies the required RPO and RTO. Promotion breaks the replication dependency and converts the secondary cluster from a read-only replica into an independent primary cluster that accepts writes, with traffic flowing from applications to the newly promoted cluster in the surviving region. Switchover is the planned equivalent used for testing or regional maintenance, where roles reverse in a controlled order: first replication is kept in sync, then writes move from the old primary cluster to the secondary cluster.
Choose promotion when the primary region is unavailable and accepting some data loss within the RPO is preferable to extended downtime. Choose switchover when both regions are healthy and the goal is zero or minimal loss with a predictable RTO for validation or failback. Because later primary cluster changes and read pools are not copied automatically, include manual configuration review and read-capacity checks in the recovery runbook so the promoted cluster can sustain production traffic immediately after role change.
Assessing performance and cost trade-offs in replication means balancing how fast writes move from the primary cluster to secondary and read replicas against how much compute and storage is provisioned to support that movement. In AlloyDB for PostgreSQL, low replication lag depends on fast regional storage and correctly sized instances, while in AlloyDB Omni (opens in a new tab) it depends on retaining enough Write-Ahead Logging (WAL) data for secondaries to catch up. Undersizing storage or instances lowers cost but increases lag and failure risk, while oversizing improves throughput and resilience at higher operational cost.
Replication lag in AlloyDB Omni is controlled by how long the primary database cluster retains WAL files for streaming replication to the secondary database cluster. Data flows from the primary server to the secondary server as a WAL stream, and the secondary server fetches past WAL segments from the primary WAL directory when it falls behind. Two parameters control this retention on the primary DBCluster: wal_keep_size keeps a minimum amount of past WAL, with a default of 100MB, and max_wal_size sets the maximum WAL growth during automatic checkpoints, with a default of 1504MB and a requirement that it be set higher than wal_keep_size.
If a secondary server falls behind by more than wal_keep_size megabytes, the primary server may remove a WAL segment that the secondary still needs. When that happens, the replication connection from the primary server to the secondary server is terminated, and downstream connections will also eventually fail. To avoid this, set WAL_KEEP_SIZE based on WAL generation rate, replication network characteristics, and replication lag between data centers. For a primary cluster with high availability (HA) enabled, both the primary and standby database servers need adequate WAL space because either server may need to supply WAL to the secondary cluster.
Exam tip: If replication breaks because the secondary fell behind, the cause is wal_keep_size being too small for the lag, and max_wal_size must always be set higher than wal_keep_size.
Storage layer performance in AlloyDB for PostgreSQL comes from a disaggregated architecture where compute and storage scale separately. Writes flow from the primary instance to regional storage, and writes are acknowledged as soon as WAL files are persisted on regional storage while block materialization is offloaded to the storage layer. Reads flow through a multi-tier cache architecture where data is automatically placed between buffer cache, ultra-fast cache, and the intelligent storage engine. This design produces low replication lag as a built-in feature and allows non-disruptive scaling with sub-second downtime for the primary and zero downtime for read pool nodes.
Because caching and offload hide disk behavior, input/output operations per second (IOPS) are not a meaningful comparison metric for AlloyDB for PostgreSQL. The primary sizing metric is transactions per second (TPS) or transactions per minute (TPM), which directly measures how much read and write work the system handles. Higher storage throughput and larger caches support higher TPS and lower lag, but they increase storage cost even when the extra throughput is not always used.
Instance sizing for throughput and cost starts with four sizing considerations that determine how much compute is needed for both primary writes and replication work:
To estimate the required size for an OLTP workload, follow an ordered evaluation where each step depends on the previous result:
HammerDB or pgbench against AlloyDB to test whether a machine size meets TPS requirements.The documentation supports concrete sizing rules that link throughput to cost. Aim for steady-state central processing unit (CPU) utilization of about 60-70% to avoid operating system scheduling overhead and leave margin for spikes, based on Transaction Processing Performance Council-C (TPC-C) guidance at roughly 65% reads and 35% writes. Start with 8 GB of random-access memory (RAM) per virtual CPU (vCPU) for shared buffers and query working memory, and note that open source PostgreSQL generally does not benefit beyond 16 vCPUs even though AlloyDB scales linearly with cores for many workloads. Adding a secondary cluster in another region and read pool instances improves geographic distribution and disaster recovery (DR), but the secondary starts with no read pool instances and does not automatically inherit later primary configuration changes, so each added instance and manual update increases operational cost and management work.
availability.numberOfStandbys parameter, and failover moves write ownership from the failed primary instance to the standby replica.wal_keep_size defaults to 100MB and max_wal_size defaults to 1504MB, and max_wal_size must be set higher than wal_keep_size or replication breaks when the secondary falls behind.Last updated