Design, migrate, and run scalable, highly available databases on Google Cloud across relational and NoSQL services, and track your readiness for the Professional Cloud Database Engineer exam.
Prepare and test your skills
Prepare and test your skills
Shared responsibility for databases on Google Cloud divides work between Google and the operator based on where the database runs. With self-managed databases, the operator keeps most lifecycle work, while Google manages only physical infrastructure, and with Google-managed services, Google automates backups, replication, and failover while the operator sets policies.
A self-managed database runs when an operator installs MySQL, PostgreSQL, or SQL Server on a Compute Engine (opens in a new tab) virtual machine (VM). Google manages the data centers, hosts, power, and physical network, but the operator manages the operating system, database software, patching, monitoring, availability design, and recovery. This model keeps the same operational concerns as on-premises operation, with added elasticity of cloud VMs but without automatic database lifecycle management.
Operator duties for self-managed databases on Compute Engine cover the full software stack from installation through auditing and access control. The operator must install and configure the database, apply operating system and database patches, monitor health and availability, perform backups, and recover from failures. Database Center (opens in a new tab) flags health issues that do not fix themselves on VMs, which shows where manual work remains:
These checks prove that patching, access control, encryption, and auditing stay operator responsibilities on Compute Engine, while Google-managed services handle much of this control-plane work internally.
Backup and recovery building blocks for self-managed databases are operator-chosen tools for copying data and rebuilding servers after failure. Backup and DR Service (opens in a new tab) is a centralized, cloud-native service that backs up and recovers self-managed MySQL, PostgreSQL, and Microsoft SQL Server in cloud and hybrid environments. Operators can also capture point-in-time disk state with Persistent Disk snapshots (opens in a new tab), which are incremental by default, stored redundantly with checksums, and restorable across zones and regions. Database backups and transaction logs can also be stored in regional Cloud Storage (opens in a new tab) buckets that replicate data across zones for durability.
A common self-managed pattern rebuilds a failed database VM without keeping a synchronized standby replica. The pattern depends on three building blocks working together, where Compute Engine runs the database server and persistent disks, managed instance groups (MIGs) detect failure and recreate the VM, and Cloud Load Balancing with internal load balancing preserves a stable database endpoint. Configuration follows an ordered workflow where each step depends on the previous image and template:
When replacement is needed, data flows from the latest snapshot to a new persistent disk, the disk attaches to the new VM created from the template, and client traffic flows from the internal load balancer to the replacement instance. Snapshot frequency directly controls RPO, while standard persistent disks require snapshot restore after a zonal failure because standard disks are zonal. A regional persistent disk variation replicates data synchronously between two zones in the same region, so snapshot restore is not required during recovery, with the choice driven by budget and recovery time objective (RTO) and RPO targets.
Automated lifecycle in Google-managed database services means backup, replication, and recovery are built into the service rather than built by the operator. If an operator uses Cloud SQL, backups are taken automatically based on the retention policy that the operator defines, with optional additional logical backups for regulatory or workflow needs. Bigtable backups is a fully managed service that saves a copy of a table schema and data and restores it to a new table later, which avoids traditional backup scripting for petabyte-scale tables. Replication between clusters in a Bigtable instance is automatically managed by Google, with each cluster holding a separate independent copy that can serve traffic after failover.
High availability ownership is the clearest difference between self-managed and Google-managed databases. A Cloud SQL instance configured for high availability (HA), also called a regional instance, runs a primary instance and a standby instance in primary and secondary zones of the configured region. In the event of primary failure, Cloud SQL fails over automatically to the standby database without requiring a change to the database endpoint IP address. A self-managed database on a Compute Engine VM provides no automatic database failover by default, so the operator must build failure detection and client redirection. Cross-zone failover often uses an observer instance that detects primary failure and orchestrates promotion, with rules tuned to avoid split-brain operation and unnecessary failover, plus an internal load balancer so the application can connect to another database if the primary is unavailable.
Exam tip: Cloud SQL in HA configuration fails over automatically with no endpoint IP change, while a self-managed database on Compute Engine requires the operator to build failover with an internal load balancer and failure detection.
The operational tradeoff between control and automation centers on who performs patching, backups, and failover:
| Model | Backups and durability | Availability and failover |
|---|---|---|
| Self-managed on Compute Engine | Operator schedules snapshots, Backup and DR Service jobs, and log copies to Cloud Storage; RPO depends on job frequency | Operator builds MIGs, health checks, internal load balancing, and observer-based failover |
| Google-managed service | Google takes backups automatically per retention policy; Bigtable manages backups and cross-cluster copies | Google replicates data and fails over automatically; operator selects regional, multi-region, or cross-region replica topology |
Choose self-managed on Compute Engine when operating-system access, custom database configuration, or specific third-party versions are required and the team can sustain patching and disaster recovery (DR) engineering. Choose a Google-managed service when automatic backups, managed replication, and automatic failover reduce administrative overhead and improve scalability and availability with less operator work.
Contrast of infrastructure control and performance requirements is the choice between hardware-level control with customer-managed software and abstracted cloud environments with Google-managed operations. Bare Metal Solution (opens in a new tab) provides purpose-built bare-metal servers in regional extensions that are co-located with Google Cloud and connected through a managed, high-performance connection with a low-latency network fabric. Spanner and other cloud-native offerings hide servers entirely and scale performance automatically, so the decision centers on licensing, compliance, and latency benchmarks.
Infrastructure ownership boundary in Bare Metal Solution sits at the server handoff between Google Cloud and the customer. Google Cloud configures and provides SAP-certified bare-metal servers and related infrastructure in the regional extension, while the customer owns ongoing software work after delivery. That customer-owned work includes:
Bare Metal Solution servers that are certified for SAP NetWeaver are available only in Bare Metal Solution regional extensions, which are not available for every Google Cloud region. Certified types are listed in SAP Note 3000343, and for new SAP NetWeaver deployments Google Cloud recommends Compute Engine C3 or C4 machine types instead of Bare Metal Solution servers.
Latency architectures for SAP on Bare Metal Solution decide where application servers run relative to the database server. Traffic flows from the application servers to the database server over either the Partner Interconnect connection to Google Cloud or the local fabric inside the same regional extension. Replication traffic between a primary site and a disaster recovery site passes through the Google Cloud network, and servers at the disaster recovery site can be used for other purposes until recovery is needed.
| Architecture | Where application servers run | Choose when |
|---|---|---|
| hybrid architecture | Application servers run on Google Cloud, database server runs in the Bare Metal Solution environment | Flexibility and lower cost are priorities |
| all-metal architecture | Application servers and database server run in the same Bare Metal Solution regional extension | Workload needs lower latency than the Partner Interconnect connection can provide, or database server requires application servers to run on the same platform |
The hybrid architecture is recommended because of its flexibility and lower cost. The all-metal architecture keeps both tiers in the same regional extension at the primary site and mirrors that layout at a second regional extension for disaster recovery.
Exam tip: Choose hybrid for lower cost and flexibility, and choose all-metal only when latency below Partner Interconnect levels or same-platform requirements force applications into the Bare Metal Solution environment.
Abstracted performance in cloud-native Spanner deployments removes hardware management while retaining high scale and consistent low latency. Spanner is a fully managed database system that performs zero-downtime software and hardware updates behind the scenes, presents the same API as a single-machine database, and scales automatically with a managed autoscaler from 1/10th of a node to tens of thousands of nodes. Key-value style Spanner databases can scale horizontally to hundreds of millions of read or write requests per second and petabytes of data while maintaining a consistent, low latency profile as the workload grows by orders of magnitude. The tradeoff for that abstraction is less hardware-level control in exchange for simpler development and operations. Spanner clients use a gRPC/HTTP2 request and response model that automatically handles connection failures, provide language-native NoSQL read-write APIs without requiring SQL knowledge, and do not require connection pooling to scale. Spanner also supports flexible JSON and BYTES types for semi-structured data and Identity and Access Management (IAM)-based access control without storing usernames and passwords in the database, so teams meet compliance and growth needs without managing servers, licensing physical hosts, or certifying hardware.
Scalability and availability mechanisms determine how a database handles more load and stays reachable during failures. In self-managed environments, the team manually configures replication, backups, storage, and failover, while in Google Cloud native managed services, replication, scaling, and failover are built into the platform and operate automatically.
Manual replication workflow for self-managed databases is the administrator-configured copying of data from a primary database server to a standby replica. A self-managed database, such as MySQL, PostgreSQL, or SQL Server installed on a Compute Engine (opens in a new tab) instance, carries the same operational concerns as an on-premises database, even though Google Cloud manages the underlying infrastructure. The administrator must choose the replication method for that specific database software, keep configurations identical between servers, and handle promotion and client redirection during failure. Setting up a standby server on Google Cloud from an on-premises primary follows an ordered workflow where each step depends on the previous one:
After failover, traffic flows from clients to the standby server in Google Cloud, and recovery requires resynchronization before clients can return. Many database systems permit only a single writeable instance, so the team may need to manually promote a read-only replica to be the writeable primary if automatic promotion does not occur. The team must also manually reconfigure DNS routing to send traffic to the system that is still up, then resynchronize databases and adjust DNS again to distribute traffic when both environments are healthy. This manual approach can achieve very small RTO and RPO values, but it increases operational effort and risk of human error.
Storage and backup options for self-managed databases on Google Cloud copy and restore disks and backups without changing database software. The centralized Backup and DR Service (opens in a new tab) backs up and recovers cloud and hybrid workloads, including self-managed MySQL, PostgreSQL, and SQL Server, to support swift data recovery. Tiered backup storage in Cloud Storage (opens in a new tab) moves infrequently accessed backup data to lower-cost classes because the requirement to access backed-up data becomes less likely over time. The tiered storage pattern for production uses three storage classes as data ages:
Snapshots and disk replication provide faster recovery than copying backup files alone. Snapshots are available across regions, allowing restoration of a disk not only within the same region but also to a different region. A variation using regional persistent disks replicates data synchronously across zones, so the snapshot restore step is not needed during recovery. Persistent Disk Asynchronous Replication provides block storage replication with low RPO and low RTO for cross-region active-passive disaster recovery at the infrastructure level rather than the workload level. The choice among standard snapshots, regional disks, and asynchronous replication is dictated by budget and required RTO and RPO values.
Native scaling and failover in managed database services means Google Cloud natively replicates data, scales throughput, and performs failover without manual server and replication configuration. Managed databases are designed for scale, so traditional backup and restore mechanisms from traditional relational database management systems are usually not available. Relational databases offer transactions, strong consistency, and sophisticated querying, but they are typically hard to scale and require careful management in a high-availability configuration, while non-relational NoSQL databases often choose availability over consistency under the CAP theorem. Native high availability differs by service, but the control plane manages placement and failover across zones and regions:
Backup and replication serve different recovery goals in managed services, as shown by BigQuery behavior. A backup is a one-time or recurrent point-in-time copy stored with the source table data, while cross-region replication is a continuous copy stored in a remote region.
| Method | Frequency | Storage location | Use cases |
|---|---|---|---|
| Backup (snapshots or Cloud Storage export) | One-time or recurrently | Same as source table data | Restore original data beyond the time travel period, recover from corruption |
| Cross-region replication | Continuously | Remote region | Create a geo-redundant replica, enable collocated joins, one-time migration between regions |
Backups protect against logical corruption because a table can be queried to a point in time within 7 days using snapshot decorators, or restored from an export that excludes corrupted data. Replication alone cannot recover from human error or corruption because corrupted data is automatically copied to the replica. BigQuery also reduces cost automatically through long-term storage, where the price of storage drops by 50 percent if a table is not edited for 90 consecutive days without degradation of performance or durability.
Exam tip: Choose point-in-time backups or snapshots to recover from data corruption, because continuous replication copies corrupted data to the replica.
Last updated