Design, migrate, and run scalable, highly available databases on Google Cloud across relational and NoSQL services, and track your readiness for the Professional Cloud Database Engineer exam.
Prepare and test your skills
Prepare and test your skills
Horizontal scaling and read capacity management is the practice of adding separate database capacity to handle more concurrent reads and higher throughput instead of only enlarging a single instance. In Google Cloud, Cloud SQL (opens in a new tab) scales reads horizontally with read replicas, while Cloud Spanner scales reads, writes, and storage together by adding nodes and processing units. Sharding is a third horizontal approach that splits one database across multiple smaller instances to scale both reads and writes.
Cloud SQL read replicas are separate instances that receive copied data from the primary instance and serve read queries from applications. Traffic flows from the application to the primary instance for writes and from the application to the read replica for reads, which lowers load on the primary and improves overall workload performance. Use read replicas when vertical scaling with more vCPU, RAM, and disk throughput is no longer sufficient and query tuning has not resolved the pressure.
Routing reads to a replica requires additional application effort because the application must be changed to connect to the replica endpoint for read queries. Frequent resizing of a Cloud SQL instance carries its own cost, so horizontal replicas are often more stable than repeated vertical changes for sustained read growth. Compute size can be scaled up as frequently as needed, but compute size should be scaled down infrequently.
Cloud Spanner compute capacity is measured in nodes and processing units (PUs), where 1000 PUs equals one node, and capacity is spread across servers in different zones. The service depends on this distributed capacity for both throughput and storage limits, and each per-zone server resource uses the data replica in its zone. For instances of 1000 PUs and smaller, Spanner allocates server resources in a single server task per zone, while instances larger than 1000 PUs use multiple server tasks per zone with one task for each 1000 PUs.
Storage allotment in Cloud Spanner is directly tied to compute capacity, so read scaling and storage scaling happen together. Instances smaller than 1 node receive 1024.0 GiB of data for every 100 PUs, and instances of 1 node and larger receive 10 TiB of data for each node. For example, a 300 GB database can run on 100 PUs until the database grows past 1024.0 GiB, after which another 100 PUs must be added or Spanner might reject writes. Spanner bills only for storage actually utilized, not for the total allotment, but peak read and write throughput still depends on instance configuration, schema design, and dataset characteristics.
Compute capacity can be increased after creation, with most requests completing within a few minutes and rare cases taking up to an hour. Compute capacity can be reduced except when stored data exceeds 10 TiB per node, so storage must be below that limit before scaling down. Instances with fewer than 1000 PUs suit smaller data sizes and workloads, but their limited compute can produce non-linear scaling and intermittent latency increases under larger workloads.
Database sharding is a horizontal scaling method that splits one database instance into multiple smaller instances using a partitioning criterion such as customer or subject. Writes and reads flow from the application to different shards based on that criterion, so both write and read capacity grow as shards are added. Sharding requires re-architecting the database and application plus data migration, which makes it much more complex than adding a read replica or adding Spanner nodes.
Unbalanced shards called hotspots can concentrate load on one shard and outweigh the benefit of sharding, so the shard key must distribute traffic evenly. Choose sharding only when a single primary or single Spanner scaling path cannot absorb write growth and the team can manage multiple instances and shard balance over time.
The three horizontal approaches differ by what they scale and what tradeoff they impose on operations and applications:
| Approach | What it scales | Key tradeoff |
|---|---|---|
| Cloud SQL read replica | Concurrent reads | Application must change to route read queries to the replica |
| Cloud Spanner node and PU increase | Reads, writes, and storage allotment | Higher compute cost and storage limit of 10 TiB per node |
| Database sharding | Writes and reads | Re-architecture, migration, and hotspot risk |
Choose a Cloud SQL read replica when the primary is read-bound and the application can separate read traffic from write traffic. Choose additional Spanner nodes or PUs when throughput, latency, or storage growth requires more distributed capacity across zones without changing the application sharding logic.
Exam tip: A Cloud SQL instance scaled down more than once in three hours gets near-zero downtime only on the first scale-down; later scale-downs in that window use regular downtime.
Proactive monitoring and automated capacity planning is the practice of using metrics and historical trends to keep database capacity ahead of demand without paying for idle resources. Cloud Monitoring (opens in a new tab) collects measurements from services and Google Cloud resources, and those measurements feed forecasts, scaling policies, and alerts that trigger action before a traffic spike causes an outage. For databases such as Cloud SQL (opens in a new tab), this loop connects workload history to storage and compute decisions so availability is protected while over-provisioning is avoided.
Forecasting workload means predicting the load a database and its surrounding stack must handle based on past and future demand signals. The forecast starts with the number of users and request rate, then adds historical load trends, seasonal variations, spikes during special events, and growth from business changes such as expansion to new geographies. That forecast flows into a capacity estimate that is shaped by deployment architecture and performance and reliability objectives, such as the size of managed instance groups (MIGs), VM machine type, and the number, type, and size of persistent disks.
Capacity estimates must include redundancy and validation rather than sizing only for the expected average. For N+1 redundancy, every component in the application stack keeps at least one redundant component beyond the minimum needed for the forecast load, so one failure does not remove required capacity. Benchmarking validates the estimate by running load tests under both normal and peak conditions to measure resource efficiency, which is the relationship between application load and CPU and memory consumed. Because efficiency can degrade under very high load and change over time, benchmarking is repeated at regular intervals and cost is checked with the Google Cloud Pricing Calculator.
Monitoring utilization means continuously collecting database and infrastructure metrics in Cloud Monitoring and turning them into dashboards and alerts that guide scaling. Cloud Monitoring provides predefined dashboards for several Google Cloud products, including a Cloud SQL monitoring dashboard and a Query Insights dashboard, plus custom dashboards that track business-specific metrics and trends. Cloud SQL uses built-in agents to collect query telemetry, so teams can see instance health, capacity utilization, and the most expensive queries from one centralized place.
Custom monitoring extends those defaults into proactive action by defining thresholds and notification paths. Teams create custom dashboards, define custom metrics, and set alert policies so notifications arrive before disk, memory, CPU, or quota limits are reached. Alerts can integrate with incident tools such as PagerDuty or Slack, while Cloud Logging, Error Reporting, Cloud Trace, and Cloud Profiler add logs, error grouping, latency tracing, and production code profiling. By observing history in the same system that sends alerts, teams can distinguish a short spike from a sustained trend and decide whether to scale, tune queries, or adjust policy.
Exam tip: Enabling automatic storage increases handles data growth, but it does not replace a buffer — keep extra headroom for peak workloads and at least 20% free disk space for maintenance, with a metrics-based alert on disk utilization.
Automated scaling means letting policies add capacity when metrics rise and remove waste when demand falls, rather than manually resizing for every change. For storage, enabling automatic storage increases lets a Cloud SQL instance grow as data grows, while still requiring a buffer because workloads tend to increase over time and maintenance needs free space. For stateless compute, the autoscaling capability of stateless MIGs adds VMs when traffic increases and removes them when need is low; stateful MIGs cannot be autoscaled, so they depend on manual planning and reservations.
Rightsizing keeps automation cost-effective by matching instance shape to observed use instead of leaving instances overprovisioned. Cloud Monitoring dashboards expose health and capacity utilization, and Active Assist turns that history into cost insights and recommendations such as identifying idle instances and reducing overprovisioned Cloud SQL instances. For predictable database needs, committed use discounts lower cost, while choosing solid-state drive (SSD) versus hard disk drive (HDD) storage trades performance for price. In general SSD is the efficient default, while HDD may fit datasets of 10 TB or more that are latency-insensitive or infrequently accessed.
Quota planning controls whether scaling is even allowed, because quotas are per-project limits that can block growth during a peak:
| Quota type | What it limits | Planning action |
|---|---|---|
| Resource quotas | Maximum resources that can be created, such as the number of regional Google Kubernetes Engine (GKE) clusters in a region | Review current limits against forecast capacity and request an increase before a spike |
| Rate quotas | Number of API requests that can be sent to a service in a specific period | Review API call volume under peak load and request an increase when throttling is likely |
Capacity planning for peaks means reserving enough assured headroom for anticipated spikes while avoiding permanent excess capacity. First, historical utilization from Cloud Monitoring is analyzed to identify trends and project future needs, then regular load testing simulates the expected peak to expose bottlenecks in CPU, memory, disk throughput, IOPS, and vCPU count. Next, quotas are checked because Google Cloud service quotas are per-project limits that can be zonal, regional, or global, and quota usage is monitored with alerts so an increase can be requested with forecasting and load-test data before traffic arrives. Only after quotas are sufficient is compute capacity reserved in a specific zone for a chosen machine type, either for one project or shared across projects.
Memory and location decisions show why this planning must happen early rather than during an outage. For performance-intensive workloads, instances should have at least 60 GB of memory and should not remain constrained on CPU or memory, because distant writers, readers, and databases add latency to inserts, updates, deletes, and reads. Scaling memory requires an instance restart, and if an instance is already at maximum memory the database must be sharded across multiple instances. Documenting the peak-event strategy and rehearsing it with drills ensures scaling behavior, reservations, and alerts respond appropriately when traffic patterns change.
Vertical scaling and resource allocation is the practice of increasing the compute resources of an existing database instance to handle more workload without adding more instances. In Google Cloud, Cloud SQL (opens in a new tab) scales vertically by changing machine type for vCPU and memory and by growing disk size and throughput, while AlloyDB for PostgreSQL (opens in a new tab) scales vertically by choosing larger primary and read pool machine types on disaggregated compute and storage. Both approaches trade higher performance and headroom for higher cost and, in some cases, brief downtime during the change.
Cloud SQL vertical scaling means increasing disk throughput, vCPU, and RAM on the same instance to resolve performance pressure quickly. A larger machine type provides more processing power and memory for queries, and higher vCPU also provides more IOPS and throughput from persistent disk SSD storage, while a disk size of 4 TB or greater provides more throughput and IOPS. This makes vertical scale-up the fastest first response when CPU, memory, or I/O is overloaded, with deeper tuning and investigation to follow in a test environment.
Scaling behavior and availability in Cloud SQL depend on direction and resource type. Compute size can be scaled up as frequently as needed, but compute size should be scaled down infrequently. Changing the memory size of a Cloud SQL instance requires an instance restart, and if the instance is already at maximum memory size, the database must be sharded across multiple instances. To retain headroom for maintenance and growth, keep at least 20% available disk space, aim for under 10,000 tables per instance, size for transaction log retention on high-write instances, and enable automatic storage increases.
Exam tip: Only the first Cloud SQL scale-down in a three-hour period benefits from near-zero downtime; subsequent scale-down events in that window experience regular downtime.
AlloyDB for PostgreSQL vertical sizing means selecting vCPU and memory for primary instances and read pool instances to meet a target transactions per second (TPS) rate with headroom. AlloyDB for PostgreSQL uses disaggregated compute and storage where writes are acknowledged as soon as write-ahead log (WAL) files persist to regional storage while block materialization is offloaded to storage. Data flows automatically between buffer cache, ultra-fast cache, and the intelligent storage engine in a multi-tier cache, so IOPS is not a useful comparison metric and TPS or transactions per minute (TPM) is the primary sizing metric.
Correct AlloyDB for PostgreSQL sizing starts from workload requirements rather than disk size alone. Before choosing a machine type, determine workload type, latency and throughput requirements, expected data size and active dataset size, and growth over time:
To estimate the required AlloyDB for PostgreSQL size from current and upcoming workload, follow this ordered workflow:
Targeting 60-70% steady-state CPU utilization avoids operating system scheduling overhead and leaves margin for spikes from client applications. During scale operations, the primary instance has sub-second downtime and read pool nodes have zero downtime, which makes vertical adjustment far less disruptive than in traditional single-node databases.
Vertical scaling decisions balance throughput gain against cost, disruption, and long-term data-model limits. Larger Cloud SQL and AlloyDB for PostgreSQL machine types handle higher TPS, higher concurrency, and larger active datasets, but they cost more and may only partly compensate for poor schema design, missing indexes, or unbalanced growth. When vertical scaling is no longer sufficient or cost-effective, read replicas can offload reads, while partitioning, archiving cold data, and sharding can distribute writes across multiple smaller instances at the cost of application and operational complexity.
| System | What scales vertically | Availability during scale |
|---|---|---|
| Cloud SQL | vCPU, memory, disk throughput and capacity on the same instance | Scale-up can be frequent; memory change requires restart; repeated scale-down loses near-zero downtime benefit |
| AlloyDB for PostgreSQL | vCPU and memory of primary and read pool instances on shared regional storage | Sub-second downtime for primary and zero downtime for read pool nodes during scale operations |
Resource exhaustion also affects reliability coverage because consistently high disk utilization without automatic storage increase, consistently high CPU utilization, or consistently high memory usage means the instance is not properly sized for its workload. Shared-core and single-zone Cloud SQL instances and overloaded instances risk exclusion from Service Level Agreement (SLA) coverage, so monitoring CPU, memory, and storage with alerts is part of resource allocation. For planning on self-managed AlloyDB Omni, start with matching CPU, RAM, and disk from the source system and use 8 GB of RAM per vCPU as the baseline ratio.
Last updated