Prepare and test your skills

Prepare and test your skills

Common database migration strategies describe how much change an organization makes when moving a database to Google Cloud. The three main choices are lift-and-shift with no schema change, replatforming to a managed service, and modernizing to a cloud-native design. Managed services improve long-term operation because replication, backup and restore, and elastic scaling are built in.
Lift-and-shift moves the existing schema and data to the cloud without redesign in the first iteration. Data flows from the existing warehouse to staging files, then to Cloud Storage (opens in a new tab), and then into BigQuery (opens in a new tab), so pipelines that feed the old warehouse keep working. Keeping the schema unchanged reduces risk, avoids retraining staff, and lets automated tools perform the transfer while proofs of concept explore cloud features in parallel.
Choose lift-and-shift when speed, low disruption, and continuity matter most. The organization can expand the data footprint in later iterations and reroute upstream feeds to BigQuery only after key use cases are tested. This approach fits tight time or staffing constraints where pipelines cannot change yet.
Replatforming moves a database to a managed service such as Cloud SQL or Bigtable (opens in a new tab) with little application change while gaining managed operations. The managed service then handles replication, durability, and scaling, so the team no longer patches servers or plans maintenance windows. This middle path fits workloads that need better reliability and scalability but cannot yet justify a full redesign.
For Cloud SQL, replication creates a replica in the new region during migration, and Database Migration Service can migrate SQL databases from different sources to Google Cloud. Database Migration Service can migrate one Cloud SQL instance to another region, but not to a different project. For Bigtable, replication copies data to the new region when the service is available there, while backup and restore creates a backup in one instance and restores it in another. A third Bigtable option exports tables to Cloud Storage and then imports them into the destination instance.
For Firestore, a regional database can move with managed export and import through a Cloud Storage bucket, or with backup and restore. Backup and restore is less expensive and more straightforward than export and import. Firestore planning must also account for App Engine, because an App Engine app in us-central or europe-west makes the Firestore database multi-regional. Choose replatforming when the workload needs managed availability, consistent low-latency reads and writes, and simpler operations without changing the data model.
Modernization redesigns the schema and application to use cloud-native capabilities for long-term transformation. In BigQuery, this means evolving from star or snowflake schemas to nested and repeated fields, which reduces joins and aligns with BigQuery internal columnar storage. BigQuery stores compressed data in Colossus using the Capacitor format and executes queries with the Dremel engine, so colocated and clustered data scans less data and aggregates faster.
Modernization is iterative and follows the initial lift-and-shift load. Teams first apply light optimizations such as partitioning and clustering on columns frequently used for filtering, then test key use cases before denormalizing further. A similar evolution applies when moving wide-column and key-value workloads to Bigtable from Apache HBase, Apache Cassandra, DynamoDB, or Aerospike, using Google-provided tools for online replication with almost no downtime or for offline migration. Choose modernization when goals include petabyte-scale analytics, single-digit millisecond latency at high throughput, elastic scaling, and lower cost per query over time.
Exam tip: Transfer the schema unchanged first to keep pipelines running, then evolve to nested fields and clustering; CSV requires a separate schema definition, while AVRO, ORC, and Parquet embed schema with the data.
Data transfer follows an ordered flow from extraction to validation and iteration. Cloud Storage sits in the middle as the staging boundary between the source environment and the Google Cloud destination, and each step depends on the previous one.
A left-to-right process flow showing Extract, Upload to Cloud Storage, Load into BigQuery, Transform, then Validate and iterate with the Data Validation Tool.
Decommissioning the source requires testing, downtime planning, and rollback readiness. New-environment tests include unit and integration tests plus traffic replication or A/B testing, while planned downtime makes a blue-green switch easier to monitor. Monitoring must be in place to detect errors and trigger rollback, and backups of the old region should be kept until the new region runs without error. The old region can later become a disaster recovery site with additional reliability design.
The choice of migration strategy depends on effort, workload characteristics, and how much operational change the organization can absorb. Transforming in the source environment is limited by existing compute and may need extra transfer time and bandwidth, while transforming on Google Cloud provides virtually limitless processing but requires building pipelines. Starting downstream with reports and dashboards shows value quickly, while facade views let upstream tables evolve independently until source tables can be retired.
A decision tree branching on speed and continuity, need for managed reliability without rewrite, and need for scale to choose lift-and-shift, replatform, or modernize.
| Strategy | Change required | Choose when |
|---|---|---|
| Lift-and-shift | Move schema and data as-is | Speed and continuity matter most and pipelines cannot change yet |
| Replatform to Cloud SQL, Bigtable, or Firestore | Move to managed replication and backup with minimal redesign | Reliability, availability, and simpler operations are needed without rewriting applications |
| Modernize to BigQuery or Bigtable design | Redesign schema and queries for columnar or wide-column storage | Analytics scale, low latency at high throughput, and long-term cost efficiency justify more effort |
Google Cloud migration tools move data from its source to its destination with less downtime and less disruption to daily operations. Database Migration Service (opens in a new tab) provides migration jobs as one execution path, while storage and data-transfer utilities handle bulk files, large offline loads, and analytics moves. Together with validation and delivery tools, these services streamline each phase from initial copy to production cutover.
Bulk data movement to Google Cloud centers on where the data starts and how large it is. Storage Transfer Service (opens in a new tab) brings data to Cloud Storage from other clouds, online resources, or local systems, including Amazon S3, Azure Blob, data lakes, and on-premises file systems. Transfer Appliance is a physical hardware appliance that you fill on-site and ship to Google Cloud for very large moves, and BigQuery Data Transfer Service automates scheduled, managed movement into BigQuery from software-as-a-service apps and from warehouses such as Teradata and Amazon Redshift.
| Tool | Source to destination | Choose when |
|---|---|---|
| Storage Transfer Service | S3, Blob, data lake, on-premises file systems, and online resources to Cloud Storage | The data can move over the network to object storage |
| Transfer Appliance | On-premises data centers to Google Cloud | The dataset exceeds 20 TB, up to 1 petabyte, and network transfer would be too slow or disruptive |
| BigQuery Data Transfer Service | SaaS apps, Teradata, and Amazon Redshift to BigQuery | The goal is refactor to BigQuery on a scheduled, managed basis |
Data flows from the source system to the Google Cloud destination without requiring the business to stop operating, which is especially important for Transfer Appliance moves. The choice depends on size and destination: network-based transfer fits cloud-to-cloud and smaller local copies, the appliance fits petabyte-scale offline loads, and BigQuery Data Transfer Service fits analytics refactoring.
Exam tip: Choose Transfer Appliance when the migration is larger than 20 TB and up to 1 petabyte; otherwise use Storage Transfer Service for online moves to Cloud Storage.
Database migration jobs move the database itself from a source such as Amazon Relational Database Service (RDS) or Amazon Aurora to a destination such as Cloud SQL (opens in a new tab). Migration guides group the execution options into built-in database engine backups, Database Migration Service migration jobs, database engine built-in replication, and third-party tools. Scheduled maintenance approaches use a planned downtime window to restore a backup, while continuous replication approaches keep the source and destination in sync and then switch production during cutover, which reduces downtime and migration-related impact.
Specialized utilities support specific database modernization paths:
The Data Validation Tool is an open source Python command-line tool based on the Ibis framework that compares source and destination tables with multi-level validation functions. Validation is a critical step in data warehouse, database, or data lake projects because it confirms that the data in Cloud SQL, Spanner, Bigtable, or BigQuery matches the source before fallback options are closed and the old environment is cleaned up.
Server migration preserves the operating system, software, and configuration along with the data. Image Import imports on-premises virtual disks, often called golden disks or golden images, into Google Cloud and uses the resulting image to create virtual machines. It supports most virtual disk file formats, including VMDK and VHD, so data flows from the on-premises virtual environment to a Compute Engine image and then to a new virtual machine.
When Image Import or Migrate to Virtual Machines is not a fit, third-party options from Google Cloud Marketplace can help. RackWare Management Module (RMM) is a fully automated enterprise-grade solution that migrates physical and virtual servers from any data center or public cloud into Google Cloud. SUREedge migrates physical and virtual environments to Google Cloud, for example for on-premises non-vSphere virtualized environments where Migrate to Virtual Machines might not fit.
Migration does not end when data arrives, because applications must be rebuilt, stored, and delivered to their target environments. Cloud Build (opens in a new tab) executes builds on Google Cloud by importing source code from Cloud Storage, Cloud Source Repositories, GitHub, or Bitbucket and producing artifacts such as Docker containers or Java archives. Artifact Registry provides a single location for managing packages and container images across the full artifact lifecycle, and Cloud Deploy automates delivery of applications to a series of target environments in a defined promotion sequence.
Containerized applications can then run on managed services such as Google Kubernetes Engine (GKE), while other workloads can deploy to serverless or flexible environments such as App Engine flexible environment, Cloud Run functions, and Cloud Run. This chain depends on the migration before it: only after data and images are validated in Google Cloud can builds and automated delivery promote the modernized system to production.
Database modernization in Google Cloud means moving a database from a legacy or self-managed environment to a managed, cloud-native service. Organizations can choose migration pathways such as managed replication, backup and restore, export and import, or Database Migration Service. They can also refactor into cloud-native architectures, especially BigQuery (opens in a new tab), Firestore, and Spanner, to improve scalability, global availability, analytics integration, and cost efficiency.
Migration pathways define how data moves from source to destination and which Google Cloud service supports the move. Managed services simplify replication, backup, and scaling, but each pathway has limits on regions, projects, and service availability that must be checked before cutover.
| Pathway | How it moves data | Google Cloud services | Key limitation or consideration |
|---|---|---|---|
| Database Migration Service | Migrates SQL databases from supported sources to Google Cloud | Cloud SQL | When the source is another Cloud SQL instance, you can migrate to a different region but not a different project |
| Replication | Creates a replica in a new region | Cloud SQL, Bigtable | Managed services simplify replication; check that the service is available in the destination region |
| Backup and restore | Creates a backup and restores it to another region or instance | Filestore, Bigtable, Firestore, Cloud SQL | Firestore backup and restore is less expensive and more straightforward than import and export |
| Export and import | Exports data to Cloud Storage, then imports it into a new instance | Bigtable, Firestore, Spanner | Firestore uses a Cloud Storage bucket; Spanner uses Dataflow templates for export and re-import |
After choosing a pathway, prepare for decommissioning the source environment. Test the new environment before switching traffic, plan downtime for cutover strategies like blue-green, and implement rollback with monitoring. Only shut down services in the first region after extended tests and a successful go-live in the new region. Keep backups of the first region for a limited time, and consider promoting the old region to a disaster recovery site.
BigQuery is a fully managed, cloud-native data warehouse that decouples data storage from compute. It stores data in Colossus using the Capacitor columnar format and uses the Dremel execution engine. Organizations can modernize by transferring schema and data, then iterating on the design to take advantage of BigQuery native features.
For an initial transfer, move the schema without changes to avoid adjusting data pipelines and to use automated tools. You can use BigQuery Data Transfer Service for Amazon Redshift and Teradata, or extract files to Cloud Storage (opens in a new tab) and load them with BigQuery Data Transfer Service, BigQuery Storage Write API, or batch load. Extract data into AVRO, ORC, or Parquet when possible because schema information is embedded; CSV requires specifying the schema separately. Upload files to Cloud Storage using Storage Transfer Service, gcloud CLI, third-party tools, compression, or Transfer Appliance. Validate the transfer with the Data Validation Tool (DVT). For transformation during load, use Cloud Data Fusion, Dataflow (opens in a new tab), Managed Service for Apache Spark, Dataprep, or third-party ETL tools. For SQL code, use batch SQL translation to migrate scripts in bulk or interactive SQL translation for ad hoc queries.
After the initial load, iterate to improve performance and cost. BigQuery supports star and snowflake schemas, but its native representation uses nested and repeated fields. Changing to nested and repeated fields reduces joins and aligns with BigQuery internal data representation. Partitioning improves query performance and cost and manages information lifecycle; BigQuery only supports partitioning by dates and integers. Clustering on columns used for filtering helps BigQuery estimate query cost, eliminate unnecessary scans, and calculate aggregates faster.
BigQuery cost and scalability depend on slots, which are units of computation. On-demand pricing charges for bytes processed, while capacity-based pricing uses slots; projects are limited to a maximum of 2,000 slots by default. Long-term storage automatically drops the storage price by 50 percent if a table is not edited for 90 consecutive days, but the price reverts if the table is edited. For availability, BigQuery replicates data across data centers depending on the dataset location: multi-region replicates across regions, and single-region replicates across zones. BigQuery integrates with analytics and AI tools such as BigQuery ML, AI APIs, Vertex AI, geospatial functions, and business intelligence solutions.
Firestore is a document database that can be migrated using managed export and import or backup and restore. Firestore locations may be bound to App Engine presence; if an App Engine app is in us-central or europe-west, the Firestore database is considered multi-regional. For a regional location, the managed export and import service uses a Cloud Storage bucket to move instances from one region to another. Backup and restore is less expensive and more straightforward than import and export for Firestore.
Spanner is a relational database that supports migration and backup through Dataflow. You can use Dataflow templates to export a Spanner database to Avro files in a Cloud Storage bucket, then re-import those files into a new Spanner database. For more controlled backups, the Dataflow connector lets you write code to read and write data to Spanner in a Dataflow pipeline, such as copying data out of Spanner into Cloud Storage as a backup target.
Exam tip: For Firestore regional migration, backup and restore is less expensive and more straightforward than managed export and import.
Different Google Cloud databases support different modernization pathways. The right pathway depends on the database type, the required data freshness, the acceptable downtime, and the destination service features. Modernization improves scalability, global availability, analytics integration, and cost efficiency by moving workloads to managed services that scale automatically, replicate across regions or zones, and connect to analytics and AI tools.
| Service | Type | Migration or modernization pathway | Key consideration |
|---|---|---|---|
| BigQuery | Cloud-native data warehouse | Schema and data transfer, ETL, SQL translation, denormalization | Decouples storage and compute; uses slots; long-term storage discount after 90 days |
| Firestore | Document database | Managed export and import via Cloud Storage; backup and restore | Location may be tied to App Engine; backup and restore is less expensive than import and export |
| Spanner | Relational database | Export to Avro in Cloud Storage and re-import via Dataflow templates; Dataflow connector for controlled backups | Supports migration and backup through Dataflow |
| Cloud SQL | Managed SQL database | Database Migration Service, replication, automated backups, cross-region replicas | Can migrate to a different region but not a different project when source is Cloud SQL |
| Bigtable | NoSQL wide-column database | Replication, backup and restore, export tables to Cloud Storage | Check destination region availability |
Lift-and-shift moves the existing schema and data to the cloud without redesign by flowing data from the existing warehouse to staging files, then to Cloud Storage, and then into BigQuery so pipelines that feed the old warehouse keep working. Choose lift-and-shift when speed, low disruption, and continuity matter most and pipelines cannot change yet, then expand the data footprint and evolve to nested and repeated fields with partitioning and clustering in later iterations.
Storage Transfer Service brings data to Cloud Storage from other clouds, online resources, or local systems over the network, including Amazon S3, Azure Blob, data lakes, and on-premises file systems. Transfer Appliance is a physical hardware appliance that you fill on-site and ship to Google Cloud for very large on-premises moves larger than 20 TB and up to 1 petabyte when network transfer would be too slow or disruptive.
Firestore backup and restore is less expensive and more straightforward than managed export and import through a Cloud Storage bucket. Choose backup and restore for moving a regional Firestore database when lower cost and simpler operations are preferred, while remembering that an App Engine app in us-central or europe-west makes the Firestore database multi-regional.
An e-commerce company plans to migrate its on-premises PostgreSQL database to Cloud SQL for PostgreSQL. Because the database supports a 24/7 online storefront, the company has the following migration requirements:
Which Google Cloud service or tool should the company use?
Storage Transfer Service
Database Migration Service
Google Cloud Migration Center
Transfer Appliance
Last updated