Design, migrate, and run scalable, highly available databases on Google Cloud across relational and NoSQL services, and track your readiness for the Professional Cloud Database Engineer exam.
Prepare and test your skills
Prepare and test your skills
A sizing baseline is the measured set of facts used to choose vCPUs, memory, disk size, and node counts for a Google Cloud database. The baseline records workload type such as online transaction processing (OLTP), online analytical processing (OLAP), or hybrid, plus required latency and throughput in transactions per second (TPS) or per minute (TPM). The baseline also records total data size, the active working set that must fit in memory, and expected growth in data and throughput over time.
Peak use matters more than average use because the target system must survive the busiest periods without errors or outages. CPU and memory analysis uses Cloud Monitoring (opens in a new tab) dashboards for health and capacity and Query Insights to find the most expensive queries that drive CPU and memory use. Disk analysis measures input/output operations per second (IOPS), throughput, and growth under normal and peak load, while network analysis measures latency between clients, writers, readers, and storage. Automated discovery completes the baseline with Migration Center for landscape assessment and total cost of ownership, migVisor for heterogeneous migrations with less than 3% collection load, and the Database Migration Assessment Tool for schemas, tables, views, functions, triggers, and procedures.
Machine and storage selection is the choice of Compute Engine shape and disk type that removes the real bottleneck without paying for unused capacity. General-purpose N4 shapes balance price and performance for mixed use, compute-optimized C3 and C4 shapes give the highest performance per core for complex queries and high-volume OLTP, and memory-optimized M3 and M4 shapes provide large memory-to-vCPU ratios for caching. For most MySQL workloads the smallest highmem shape that fits the working set in RAM is preferred, with about 8 GB of RAM per vCPU, so steady-state reads flow from memory to the database without disk access.
Storage tiering matches disk IOPS and throughput to workload shape because disk speed controls database responsiveness. IOPS counts small read and write requests per second and controls OLTP responsiveness, while throughput measures data moved per second and controls OLAP scans, warehousing, and backups.
| Disk type | Performance behavior | Best use |
|---|---|---|
| SSD Persistent Disk | Performance scales with disk size and VM vCPUs | Workloads where larger disk and VM raise speed |
| Hyperdisk Balanced | IOPS and throughput scale independently of size | Production data and logs tuned to actual need |
| Hyperdisk Extreme | Very high configurable IOPS with lowest durable latency | Mission-critical OLTP such as finance or ecommerce |
| Local SSD | Highest IOPS from disks on the same host with no durability | Temporary data such as tempdb only |
Traffic flows from the virtual machine (VM) to Local SSD on the same host for very low latency, but Local SSD does not survive host failure. Separate disks for data and logs isolate sequential log writes from random data reads, and Hyperdisk Balanced allows provisioned IOPS to follow writes and maintenance rather than disk size.
Capacity mapping translates the baseline into Cloud SQL (opens in a new tab) instance size, AlloyDB for PostgreSQL machine size, or Cloud Spanner processing units with room for growth. Cloud SQL scales vertically on one instance with vCPU, memory, and SSD persistent disk and scales out for reads with read replicas, where disk performance depends on both disk size and vCPU count. Cloud SQL instances should keep at least 20% free disk space, stay under 10,000 tables per instance to limit upgrade time, and enable automatic storage increases for growth.
AlloyDB for PostgreSQL uses disaggregated compute and storage with multi-tier caching across buffer cache, ultra-fast cache, and intelligent storage. Writes are acknowledged when write-ahead log files persist to regional storage, while block materialization is offloaded to storage, so TPS and TPM are the meaningful sizing metrics rather than IOPS. Target about 60-70% steady-state CPU utilization to leave margin for spikes, and deploy as a zonal single node or a highly available two-node instance with optional read pools and a secondary cluster for distributed reads or disaster recovery (DR).
Cloud Spanner capacity is provisioned in processing units (PUs), where 1000 PUs equals one node, plus the storage allotment tied to that compute. Spanner bills for storage actually used but may reject writes if data exceeds the allotment for the current compute size.
| Spanner size | Storage allotment per database |
|---|---|
| Smaller than 1 node (1000 PUs) | 1024.0 GiB for every 100 PUs |
| 1 node and larger | 10 TiB for each node |
Spanner Omni maps demand to host VM or Google Kubernetes Engine (GKE) node resources plus durable block storage. Each deployment uses a dedicated persistent attachable SSD with an ext4 file system, where local disks are not supported. Allocate 500 GB of storage, 500 IOPS, and 30 MB per second per vCPU, with a minimum of 4 vCPU and 16 GB of RAM using zonal pd-ssd or Hyperdisk Balanced storage.
Exam tip: For AlloyDB for PostgreSQL use TPS and TPM for sizing parity, not IOPS, because disaggregated storage and multi-tier caching make IOPS incomparable to other systems.
Growth and availability planning enlarges the baseline so the database stays reliable as demand rises and components fail. A complete forecast combines historical trends and request rates, seasonal variations and event spikes, and business growth such as expansion to new geographies. Traffic flows from users and applications to the database tier, so the forecast sets the CPU, memory, throughput, and IOPS that later sizing depends on. Validation sustains the model through load tests at normal and peak conditions, repeated benchmarks, 20% free storage with utilization alerts, transaction log retention planning, and quota checks for assured zonal capacity.
High availability (HA) overhead is extra N+1 capacity so traffic keeps flowing during a zone failure, with designs using three zones per region and overprovisioning by about a third. Cloud SQL in high availability configuration uses synchronous replication between zones in one region with automatic failover without changing the endpoint. Disaster recovery overhead covers regional outages through cross-region replicas, multi-region snapshots, and backups in dual-region or multi-region Cloud Storage buckets, where data flows from the primary region to the secondary region through asynchronous replication.
| Approach | Scope | Replication and recovery |
|---|---|---|
| High availability (HA) | One region across zones | Synchronous replication with zero or near-zero recovery time and recovery point objective (RPO) |
| Disaster recovery (DR) | Across regions | Typically asynchronous replication with defined recovery time objective (RTO) and RPO and manual or semi-automated failover |
Recovery objectives turn business needs into choices about zonal, regional, and multi-regional deployment scope. Recovery Time Objective (RTO) measures downtime until the system runs again after a disaster, while Recovery Point Objective (RPO) measures how much committed data can be lost. In Google Cloud, RTO and RPO are defined by the scope of loss, meaning a zone outage, a region outage, or a multi-region outage. Wider distribution places copies of data and compute on different sides of those failure boundaries, which directly improves RTO and RPO.
Business-critical operations that need strict RTO and RPO can depend only on data plane components that continuously process operations. If a critical operation needs the management plane during an outage, such as a virtual machine creation API or an identity permission update, recovery stalls even when data remains intact. Tiered goals reflect this cost, from Tier 1 systems needing near-zero RTO and RPO to Tier 3 systems tolerating hours of downtime and data loss window.
Deployment scope is the choice of how many zones and regions hold copies of data and compute to survive failures. A zonal deployment hosts resources inside a single zone, so an interruption in that zone can affect all resources in that zone. Compute Engine virtual machines and Persistent Disk are zonal resources with a platform availability target of 99.9% and an RTO of hours or days, requiring operator-built failover with health checks, instance templates, and managed instance groups to restore elsewhere.
A regional deployment redundantly deploys resources across multiple zones in one region, so the resource serves from another zone if one zone is interrupted. AlloyDB Omni (opens in a new tab) availability architectures show the progression from Standard Availability with backup-only protection, to Enhanced Availability with zonal replication, to Premium Availability with zonal plus regional replication for mission-critical needs. Cloud SQL configured for high availability, regional managed instance groups, and regional Spanner instances target 99.99% availability with near-zero RTO when data is replicated synchronously between primary and standby copies. Regional protection stops at the region boundary, so a regional outage requires activating a passive cross-region replica and promoting it to primary.
A multi-regional deployment distributes resources within and across regions, so service continues from surviving regions when an entire region fails. Spanner multi-region instances, dual-region and multi-region Cloud Storage buckets, multi-cluster Bigtable instances, and Firestore multi-region locations target 99.999% availability with near-zero RTO when replication is synchronous. Spanner regional configurations maintain three read-write replicas in different zones, while Spanner multi-region configurations add two read-write regions plus a witness region holding a witness replica.
| Strategy | Availability target | Zone outage robustness | Region outage robustness |
|---|---|---|---|
| Zonal | 99.9% | RTO of hours or days with manual restore or failover | RTO of hours or days with restore from backup in another region |
| Regional | 99.99% | Near-zero RTO with synchronous replication and automatic failover | RTO of hours or days by activating a replica in another region |
| Multi-regional | 99.999% | Near-zero RTO with synchronous replication from surviving zones | Near-zero RTO with synchronous replication from surviving regions |
Replication mode is the write pattern that controls whether speed or data safety is prioritized across distance. Synchronous replication waits for replica confirmation before reporting a transaction as committed, which provides RPO=0 for committed data but adds a cross-zone or cross-region round trip to every write. Regional Persistent Disk, Hyperdisk Balanced High Availability, and Cloud SQL high availability use synchronous replication inside a region for zero data loss. Asynchronous replication commits on the primary before replicas confirm, so writes stay fast but recent changes may be missing elsewhere, creating replication lag and possible data loss during a regional outage.
Latency grows with distance because every synchronous confirmation must travel from the primary to each replica and back. Zonal writes complete without cross-zone waits, regional writes add a cross-zone round trip, and multi-regional writes confirmed across continents add much larger delays. This forces a choice between strong consistency with higher write latency and lower latency with risk of stale reads or loss. Cost and complexity also rise with distribution because more resources are provisioned and more cross-location traffic flows between sources and destinations.
Exam tip: Regional deployment protects against zone outages but not region outages, and synchronous replication provides RPO=0 at the cost of higher write latency.
Maintenance planning times upkeep and alerts so planned work does not become an outage. Maintenance windows are scheduled periods that align disruptive upkeep with low-traffic periods for the application. Core upkeep tasks to schedule during a window are index defragmentation for heavily updated tables, statistics updates to keep the optimizer choosing efficient plans, vacuum analyze to clean tables and update visibility, and reindex to rebuild inefficient access paths. Preparing Cloud SQL instances for smooth maintenance means keeping at least 20% available space, keeping table count under 10,000 tables per instance, sizing for transaction and binary log retention, and enabling automatic storage increases.
Maintenance mode for AlloyDB Omni in Kubernetes keeps a database pod in a running state while repair work proceeds by disabling liveness and startup probes on the database container. The ordered sequence is first to verify probes with kubectl get po in the namespace, then to disable the probes to enter maintenance mode, and only after work is complete to re-enable probes so health reporting flows again from the database container to Kubernetes. Rescheduling rules protect availability by moving recurring upkeep away from peak demand and by postponing a second scale-down until after a three-hour window, since only the first scale-down in three hours gets near-zero downtime.
Incident response follows an ordered escalation where low-effort actions are tried before disruptive re-architecture:
Secure connectivity is how applications reach Google Cloud databases over private paths while stored data and network traffic stay protected. Customer-Managed Encryption Keys (CMEK) in Cloud Key Management Service (KMS) protect data at rest, while Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protect data in transit. Virtual Private Cloud (VPC) networks carry private database traffic, Private Service Access links customer networks to Google-managed networks, and identity and perimeter controls limit who can connect and where data can move.
Protecting stored data with customer-managed encryption means the database encrypts data at rest with a key in the customer Cloud KMS key ring. The database service depends on Cloud KMS for every encrypt and decrypt operation, so if the key is disabled, destroyed, or its permission is revoked, the database cannot read its own stored data. CMEK is applied at creation time with a different enforcement point per resource: an AlloyDB cluster references the key in cluster configuration, a Cloud SQL (opens in a new tab) instance references the key in the kmsKeyName field, and a migration destination must already be CMEK-enabled. A Cloud SQL instance cannot gain CMEK after creation, so the ordered fix is to create a new CMEK-enabled instance, move data over, and only after that delete the original.
The Cloud KMS key lifecycle moves through ordered states from generation with an approved algorithm and protection level, to distribution through CMEK grants to authorized service accounts, to storage separate from data with limited access and monitoring, to retirement at the end of its cryptoperiod or on suspected compromise. Requiring SSL and TLS means traffic flows from the database client across the network boundary to the database server only when both sides agree on TLS 1.2 or higher with trusted, valid, unexpired certificates. Controls for AlloyDB and other databases require SSL on the instance and enforce SSL for incoming connections, blocking fallback to insecure versions or algorithms.
Private isolation with Private Service Access gives a Cloud SQL instance a private IP address inside a VPC so traffic stays on Google internal network. Private Service Access connects the customer VPC network to the Google-managed service network for Cloud SQL through internally managed VPC Network Peering. Traffic flows from resources in the customer VPC network to internal IP addresses of Cloud SQL resources in the Google-managed VPC network, and the boundary sits between those two peered networks. Private IP connections work across regions but do not work from legacy networks that lack peering support.
Setting up a new project for private IP requires an ordered network foundation before instance creation:
sqladmin.googleapis.com, compute.googleapis.com, and servicenetworking.googleapis.com.VPC_PEERING, commonly with --prefix-length=16.servicenetworking.googleapis.com using the allocated range.Reaching private instances from multiple VPC networks is difficult because VPC Network Peering is not transitive, so routes are shared only between two directly peered VPCs. To connect one private Cloud SQL instance to multiple VPCs, export private service routes with custom advertised routes, bridge traffic with an intermediate proxy such as SOCKS5, or forward private traffic with Cloud SQL Auth Proxy as a service. Shared VPC extends the private model across host and service projects, where the service project owns the instance and the host project owns the network, but an existing instance cannot later be assigned a private IP in a Shared VPC network.
Secure connection paths describe where the database client must run so private traffic never leaves the VPC boundary:
| Connection path | Where the client runs | Requirement to stay private |
|---|---|---|
| Direct private IP | Compute resource inside the same peered VPC network | Client uses the instance private IP within the VPC boundary |
| Cloud SQL Auth Proxy | Proxy process on a resource with access to the same VPC network | Start the proxy with -ip_address_types=PRIVATE when both IP types are enabled |
| Serverless VPC Access | App Engine standard, Cloud Run, or Cloud Run functions | Application connects through Serverless VPC Access without the proxy |
A Cloud SQL instance can use private IP at creation or added later, with public and private IP enabled together without affecting each other. Configuring private IP on an existing instance or changing its network restarts the instance for a few minutes, private IP cannot be removed after configuration, a read replica inherits status from its primary, and the address is static.
Exam tip: A Cloud SQL instance cannot gain CMEK after creation and requires migration to a new instance, while SSL for connections can be enforced on an existing instance.
Session pooling is the reuse of ready-to-use database sessions so many operations share a smaller set of connections. Creating a new connection requires a handshake and round trip that adds latency and consumes memory and CPU. A session pool keeps backend connections alive, returns an existing session for an operation, and reclaims it afterward, which lowers latency, reduces overhead, and prevents connection exhaustion.
Reusing Spanner sessions avoids paying session-creation cost for every operation because each Spanner session provides context for one database and requires a round trip to create. Spanner sessions are intended to be long-lived and each regular session executes a single transaction at a time, so the pool must hold enough sessions for concurrent work. Multiplexed sessions run multiple concurrent transactions on one Spanner session, where requests flow from the application into the same session context and onward to the Spanner database service in parallel. A gRPC channel pool is the transport layer that carries session traffic, and tuning it increases capacity so requests do not queue and raise p95 and p99 tail latency during high traffic.
| Pooling approach | What is shared | Performance effect |
|---|---|---|
| Session pool | Long-lived sessions across operations in one database | Spreads session-creation round-trip latency across many operations |
| Multiplexed sessions | One session across concurrent transactions | Reduces backend memory and CPU overhead for concurrent work |
| gRPC channel pool | Transport channels across session traffic | Reduces p95 and p99 tail latency from congestion |
Preventing exhaustion from ephemeral serverless scale-out means controlling how many Cloud Run instances and requests share limited database connections. A Cloud Run instance serves multiple simultaneous requests up to a configurable maximum concurrency with a default of 80, and each instance that opens its own connections multiplies total demand. Operators must satisfy the rule that maximum instances times connections per instance is less than the maximum connections allowed by the database. For Cloud SQL workloads the same principle uses HikariCP maximumPoolSize to limit total connections and minimumIdle to keep warm idle connections, with small short transactions preventing pooled connections from being held too long.
Auditing policy decides which administrative and data-access events to record, how long to keep them, and who can read them. Cloud Audit Logs records who did what, where, and when on Google Cloud resources, with entries written by services to Cloud Logging for each project, folder, and organization. Log entries are immutable once written, and for managed databases such as Cloud SQL, Spanner, and Bigtable the same categories separate control-plane administration from data-plane access.
| Log category | What it records | Default state |
|---|---|---|
| Admin Activity audit logs | User-driven writes to configuration or metadata | Always enabled, cannot be disabled |
| Data Access audit logs | User-driven reads of configuration and reads or writes of user data | Disabled by default, requires explicit enablement |
| System Event audit logs | Google system-driven changes such as maintenance or failover | Always enabled, cannot be disabled |
In Cloud SQL the category follows the permission type, where ADMIN_WRITE generates Admin Activity logs and ADMIN_READ, DATA_READ, and DATA_WRITE generate Data Access logs. Choosing granularity means selecting in the IAM AuditConfig object whether ADMIN_READ, DATA_READ, and DATA_WRITE are recorded, where enabling full DATA_READ and DATA_WRITE gives query visibility at the cost of much larger volume. Holders of the Logs Viewer role can read Admin Activity and System Event logs but not Data Access logs in the _Default bucket, while Private Logs Viewer adds Data Access access.
Retaining and exporting telemetry keeps a tamper-resistant record beyond Cloud Logging retention. Audit telemetry flows from Google Cloud services into Cloud Logging and from Cloud Logging through log sinks to BigQuery for analysis, Cloud Storage for archival, or Pub/Sub for streaming. Retention policies with Bucket Lock prevent early deletion, while VPC Service Controls places a virtual boundary around resources so audit and database data cannot be copied outside even by an IAM-authorized caller.
Continue with this study guide: Determine how applications will connect to the database
Database ownership is the division of patching, backups, replication, and failover between the operator and Google. Self-managed databases on Compute Engine (opens in a new tab) leave that work to the operator, while Google-managed services automate it behind a service API. On Compute Engine, Google manages data centers, hosts, power, and physical network, while the operator installs MySQL, PostgreSQL, or SQL Server on a virtual machine and owns the operating system, patching, monitoring, availability design, and recovery.
Operator recovery often rebuilds a failed database VM without a synchronized standby, where data flows from the latest snapshot to a new disk and client traffic flows from internal Cloud Load Balancing to the replacement. The ordered build is to create a VPC network, build a custom image with a startup script that recreates disks from snapshots, create an instance template, create a managed instance group with target size 1, configure health checking with Cloud Monitoring, configure internal load balancing for a stable endpoint, and schedule snapshots to define RPO. Bare Metal Solution keeps hardware-level control for licensed SAP and Oracle workloads in regional extensions connected through Partner Interconnect or local fabric, with hybrid architecture placing apps on Google Cloud and all-metal architecture placing apps and database together for lowest latency.
| Model | Backups and durability | Availability and failover |
|---|---|---|
| Self-managed on Compute Engine | Operator schedules snapshots, Backup and DR Service jobs, and log copies | Operator builds managed instance groups, health checks, and internal load balancing |
| Google-managed service | Google takes backups per retention policy | Google replicates data and fails over automatically by topology choice |
Spanner shows abstracted scale with zero-downtime updates, a single-machine API, managed autoscaling, and support for very high request rates and petabytes with consistent latency. Cloud SQL in high availability configuration runs primary and standby in two zones and fails over automatically without changing the endpoint IP address.
Data shape and consistency needs decide whether a rigid relational schema or a flexible NoSQL model fits. Structured data has stable fields and relationships, while semi-structured and unstructured data varies in shape or has no fixed fields. AlloyDB for PostgreSQL enforces rigid schema-on-write for structured transactional records with transactions, strong consistency, referential integrity, and joins. Flexible stores such as Firestore, Datastore, Bigtable, and Memorystore avoid requiring full schema upfront and suit variable documents, wide-column key-value data at terabyte to petabyte scale, and in-memory access.
| Engine | Best-fit data shape | Schema and consistency approach |
|---|---|---|
| AlloyDB for PostgreSQL | Structured records with stable relationships | Rigid schema-on-write with transactions and joins |
| Firestore and Datastore | Variable documents for web and mobile | Flexible document schema with automatic scaling |
| Bigtable | Massive operational and analytical key-value data | Flexible wide-column schema for large scale |
| Spanner | Growing workloads needing correctness plus flexible values | Relational schema plus JSON and BYTES with ACID transactions |
Vector embeddings support meaning-based retrieval by representing text, images, or video as long number lists where similar meanings sit close together. A query converts input into an embedding and returns nearest neighbors by COSINE, L2_SQUARED, or DOT_PRODUCT distance for recommendations, semantic search, and retrieval-augmented generation (RAG). BigQuery generates embeddings with AI.EMBED and AI.GENERATE_EMBEDDING beside analytical data, while Cloud SQL for MySQL stores vectors in VECTOR columns under ACID and InnoDB replication. Exact K-nearest neighbors scans all data for accuracy, while approximate nearest neighbors checks only nearby index partitions for speed at some recall cost.
Database cost analysis compares billing models, storage and network charges, and managed versus self-managed labor. Total Cost of Ownership includes hardware, licenses, power, cooling, support, network use, and engineering time, not just instance size. On-demand pricing bills actual use for testing and spiky demand, while committed use discounts (CUDs) lower rates for one-year or three-year commitments for Compute Engine resources or product spend. Serverless consumption scales cost with traffic, Spot VMs cut cost by more than 80% for preemptible batch work, and sustained use discounts apply after continuous-use thresholds.
Storage, networking, and lifecycle costs determine long-term spend for data at rest and in motion. Dual- and multi-region buckets cost more than single-region buckets, Compute Engine prices differ by region, and Google Cloud charges for traffic between regions as egress. Closely related compute and storage should sit in the same region to avoid egress, and lifecycle management with Cloud Storage Object Lifecycle Management and Autoclass moves infrequent data to cheaper tiers. Tiered backup storage moves aging data from Nearline to Coldline to Archive for lowest storage cost with added retrieval cost.
| Option | What is paid for | Who owns operations |
|---|---|---|
| Cloud SQL | CPUs, memory, provisioned storage, region, egress, and IP use with license included | Google owns hardware, maintenance, backups, and failover |
| Compute Engine self-managed | VM, disks, network, plus separate database licenses | Team owns patching, upgrades, availability, backups, and monitoring |
Exam tip: Avoid inter-region egress by co-locating compute with the data it uses, and choose Local SSD only for temporary data because it has the highest IOPS but no durability.
Application and database dependencies link network placement, connection behavior, and security controls together. Distance adds latency and egress cost, connection growth can overload the database, and identity boundaries decide what data can move. Virtual Private Cloud (VPC) connects tiers, with a baseline placing load balancers, Compute Engine VMs, and database VMs in one VPC network and subnet for the shortest path. Private-only VMs reach Google APIs through Private Service Connect or Private Google Access, and outbound traffic flows through Cloud NAT or Secure Web Proxy.
The ordered request path names each hop from edge to data:
A connection pool is an application-side set of pre-created connections that requests reuse to reduce latency. Each service instance creates its own pool, so the database total equals the sum across instances and uncontrolled scale-out overloads the database. Identity control uses service accounts bound by Identity and Access Management (IAM) roles with inheritance and least privilege, plus Cloud SQL IAM database authentication that flows from application identity through IAM to the database. VPC Service Controls creates a perimeter where data flows only between inside resources and outside requests are denied even when IAM authorizes the caller.
Regulatory and policy compliance proves who accessed data, who holds encryption keys, and where data stayed. Cloud Audit Logs in Cloud Logging answers who did what, where, and when, with activity flowing from services to Cloud Logging and then to a log sink for retention. Coverage includes Admin Activity logs for control-plane changes, Data Access logs for reads and writes, Cloud Identity logs for logins, and VPC Flow Logs and firewall logs for surrounding activity, while Access Transparency records Google personnel access. Evidence lifecycle runs from generation to export to retention with Bucket Lock against early deletion to restricted access, with Sensitive Data Protection discovering and redacting identifiers.
Encryption and key control decide who can revoke access to data at rest. Default Google-managed encryption rotates keys automatically, while customer-managed keys in Cloud Key Management Service let the customer own lifecycle and revocation. A CMEK database can only be created with its key at creation time and cannot be converted later, with creation flowing from service identity to key material to permission to database. If permission is removed or the key is disabled or destroyed, decrypt calls fail until access is restored. Data residency keeps data inside legal jurisdictions through region selection, where application-to-database flows never leave approved regions through replication, export, or tooling. Organization policies for service usage and location restriction can block disallowed services or regions, quotas constrain failover capacity in the recovery region, and recovery must depend only on the data plane during an outage.
Multi-technology design moves, replicates, or queries data across engines and clouds without fragile copies. Data flows from each source to the migration system for extraction and then to each target for insertion with optional transformation, with independent databases moving in parallel. Log-based change data capture (CDC) such as MySQL binary-log reading preserves order with the least source load, while differential querying polls timestamp fields, adds query load, needs schema changes, and handles deletes with flags. Cardinality can redesign architecture from 1:1 copy to n:1 consolidation, 1:n distribution to regional databases, or n:m rebalancing, with active-passive writes limiting conflict risk and active-active writes requiring resolution.
Hybrid and multi-cloud architectures distribute processing with private networking and governed access. The tiered hybrid pattern runs a stateless frontend in Google Cloud while the data-heavy backend stays on-premises, with client requests flowing to the frontend and then back to the backend through an API gateway. Secure connectivity uses VPC isolation, Cloud Router with Border Gateway Protocol over Interconnect and VPN, Cloud NAT for outbound access, and Cross-Cloud Interconnect for private Google Cloud to Amazon Web Services access. Federation avoids movement by querying data where it lives through the open Apache Iceberg format with Parquet files.
Federation and external tables let BigQuery analyze data without migrating it first. Federated queries send a statement from BigQuery to AlloyDB, Spanner, or Cloud SQL through a BigQuery Connection API connection and return a temporary table using EXTERNAL_QUERY in the source dialect. External tables store metadata in BigQuery while data stays outside and use standard BigQuery syntax, while BigLake tables use access delegation through a service account for Cloud Storage and cross-cloud stores.
| Method | How querying works | Representative sources |
|---|---|---|
| Federated queries | EXTERNAL_QUERY in source dialect returns temporary table | Spanner, Cloud SQL, AlloyDB |
| BigLake tables | BigQuery syntax against table reference with delegation | Cloud Storage, S3 and Azure via Omni |
| External tables | BigQuery syntax against table reference | Cloud Storage, Bigtable, Drive |
Generative AI use of databases stores embeddings beside business data, serves similarity search at scale, and grounds large language model (LLM) answers in retrieved facts. Retrieval-Augmented Generation (RAG) retrieves real-time context before generation to reduce hallucinations. A grounding path supplies prompt context as embeddings for similarity search or as standard query results from Cloud SQL or Firestore, with Gemini optionally using Google Search grounding for internet facts.
| Option | Best for | How it relates to data |
|---|---|---|
| Vertex AI Search | Fully managed search without exposing vectors | Indexes sources and serves grounded results |
| Vertex AI RAG Engine | Managed orchestration with LlamaIndex-like interface | Retrieves from vector stores including Feature Store |
| Vertex AI Vector Search | Low-latency large-scale serving | Dedicated index for approximate nearest-neighbor search |
| Cloud SQL with Vertex AI | Operational data already in SQL | Calls models from SQL with low latency |
Vector storage in AlloyDB AI keeps embeddings in a vector column beside business rows so one query combines similarity with SQL filtering and reranking. The serving flow runs from user request through a frontend, to query embedding with the same model and parameters as ingestion, to semantic retrieval in AlloyDB combined into a contextualized prompt, to LLM generation, to logging in Cloud Logging and BigQuery, to responsible AI filtering before delivery. Ingestion flows from a Cloud Run function that chunks new data, to an embedding model, to AlloyDB for PostgreSQL with the pgvector extension.
| Index | Best fit | Documented performance |
|---|---|---|
| ScaNN | Large-scale and filtered search where build time and memory matter | Up to 10x faster creation, 4x faster queries, 10x faster filtered search than HNSW, 3x less memory |
| HNSW | Higher-dimensional data fitting largely in memory | Columnar-accelerated HNSW up to 4x faster than standard HNSW |
Scaling and securing AI workloads keeps retrieval fast without exposing embeddings. AlloyDB clusters provide high availability by default with active and standby nodes in two zones, Vertex AI Vector Search endpoints autoscale nodes with queries per second, and Pub/Sub flow control absorbs spikes before they reach the database. Governance uses least-privilege IAM, row-level and column-level security, SSL-only connections with TLS 1.3, VPC Service Controls perimeters, private IPs, default encryption with optional CMEK, and regional residency controls.
Continue with this study guide: Evaluate appropriate database solutions on Google Cloud
Last updated