Prepare and test your skills

Prepare and test your skills

Core storage and database services are chosen by data shape and whether the work is operational or analytical. Cloud Storage (opens in a new tab) holds unstructured objects such as files, images, backups, and static website assets in buckets. BigQuery (opens in a new tab) holds large analytical datasets for queries, reporting, and machine learning, while Cloud SQL (opens in a new tab) and Spanner hold operational records for day-to-day transactions in relational tables.
Cloud Storage stores each object with a globally unique ID plus system and user metadata in a flat hierarchy inside buckets. Buckets replicate data across regions for durability, support versioning and soft delete for rollback after overwrites, and connect to other services for hosting, delivery, and events. BigQuery separates storage from compute so each scales independently, with compressed columnar storage in Colossus and query execution in Dremel using slots that include CPU and RAM. Cloud SQL provides regional MySQL, PostgreSQL, and SQL Server for row-level transactions, while Spanner provides global-scale relational transactions that stay available across regions.
| Service | Data structure | Choose when |
|---|---|---|
| Cloud Storage | Objects in buckets | Data is files, media, backups, or static web assets |
| BigQuery | Columnar analytical tables | Work is large-scale queries and reporting |
| Cloud SQL | Regional relational tables | Work is operational transactions with a specific engine |
| Spanner | Global relational tables | Transactions must scale and stay available globally |
Exam tip: Choose Cloud SQL or Spanner for operational transactions on current records, BigQuery for analytical queries over large datasets, and Cloud Storage for objects such as files and backups.
Block, shared file, and object storage differ in how application servers attach to data and survive zone or region outages. Compute Engine provides virtual machines where teams control the operating system, Google Kubernetes Engine (GKE) runs containers with automated deployment and scaling, and Cloud Run runs code serverless with minimal management. Persistent Disk provides block volumes attached to a single virtual machine, Filestore provides shared files mounted by many virtual machines, and Cloud Storage provides objects accessed from inside and outside Google Cloud.
Data flows from the virtual machine to its storage and back, and replication decides what survives an outage. Zonal Persistent Disk stores data in one zone, while regional Persistent Disk replicates synchronously across two zones so every write completes in both places. Filestore (opens in a new tab) regional instances replicate synchronously across three zones for shared configuration and logs, while NetApp Volumes provides continuously available SMB storage for SQL Server failover clusters. Regional Cloud Storage buckets replicate synchronously across zones, while dual-region and multi-region buckets store data redundantly in two or more locations with synchronous metadata and asynchronous data.
| Storage option | Replication behavior | Choose when |
|---|---|---|
| Persistent Disk regional | Synchronous across two zones | VM-attached storage must survive a zone outage |
| Hyperdisk Balanced | Tunable IOPS and throughput | Performance must be tuned for demanding workloads |
| Filestore regional | Synchronous across three zones | Many VMs must share the same files |
Data pipelines move raw events through ingestion, transformation, and governance before analysis. Pub/Sub (opens in a new tab) ingests streaming events, Dataflow runs unified batch and streaming pipelines, Dataproc runs managed Spark and Hadoop clusters, and Dataplex governs data across systems. The services compose in order: Pub/Sub captures events, Dataflow transforms them, and Dataplex tracks ownership and quality.
A process flow showing event producers publishing to a Pub/Sub topic, Dataflow processing the events through three stages (source, transformation, sink), and results written to BigQuery and Cloud Storage, with Dataplex providing governance over the pipeline's data.
A publisher sends a message to a Pub/Sub topic, and each subscription receives its own copy so one slow consumer does not block others. A Dataflow pipeline written with Apache Beam then flows through three stages in order: first source reads from Pub/Sub or Cloud Storage, then transformation filters or aggregates, then sink writes to BigQuery or Cloud Storage. Dataproc fits existing Spark jobs that should move without rewriting, while Dataplex organizes lakes, zones, and assets and applies metadata and quality checks without moving data itself.
The four storage models divide data work by structure, scale, and durability needs. Cloud Storage (opens in a new tab) provides object storage, Cloud SQL and Cloud Spanner provide relational storage, Firestore and Cloud Bigtable provide NoSQL storage, and BigQuery provides analytical storage. All are managed services with built-in high availability, backups, and resilience backed by service agreements.
Replication behavior depends on bucket or database placement. Regional data is replicated synchronously across zones in one region, which supports near-zero recovery time objective (RTO) and recovery point objective (RPO). Dual-region and multi-region object data stores metadata synchronously across regions and replicates data asynchronously, with turbo replication providing a 15-minute RPO for dual-region buckets. Spanner extends synchronous protection further by supporting an active-active topology across two regions, so traffic flows to both regions and data stays replicated between them.
| Model | Example services | Durability and scale emphasis |
|---|---|---|
| Object | Cloud Storage | Low-cost replication across zones or regions |
| Relational | Cloud SQL, Cloud Spanner | Synchronous replication, including multi-regional active-active for Spanner |
| NoSQL | Firestore, Cloud Bigtable | Elastic scale for high-volume operations |
| Analytical | BigQuery | Managed scale for repeated queries |
Exam tip: Regional data is replicated synchronously across zones, while dual-region and multi-region metadata is written synchronously and data is replicated asynchronously.
Each of the six data services matches a different workload pattern for transactions, operations, or analytics. Cloud Storage (opens in a new tab) fits unstructured retention such as images, backups, and data lake files. Cloud SQL fits regional transactions with MySQL, PostgreSQL, or SQL Server, while Cloud Spanner fits globally consistent transactions with horizontal scale and strong consistency across regions.
A decision tree branching on data type and workload to choose among Cloud Storage, BigQuery, Cloud SQL, Spanner, Bigtable, and Firestore.
Cloud Bigtable (opens in a new tab) is a wide-column NoSQL database for massive operational throughput with high throughput and low latency, but it does not support complex joins or multi-row transactions. BigQuery is a serverless warehouse with columnar storage and SQL for large-scale analytics, with data flowing from Cloud Storage and software-as-a-service apps into BigQuery. Firestore, also called Cloud Firestore, is a document NoSQL database with schema flexibility, real-time sync, offline support, and automatic scaling for mobile, web, and IoT apps, where data flows from one client to Firestore and then to other connected clients.
Common business scenarios show which service fits archiving, transactions, real-time operations, and reporting. Archiving needs cheap retention, transactions need relational correctness, real-time operations need low-latency scale, and reporting needs analysis over large data. Cost, performance, and functional fit together determine the choice.
| Business scenario | Best-fit service | Why it fits |
|---|---|---|
| Archiving | Cloud Storage | Low-cost object storage for files and long-term retention |
| Ecommerce transactions | Cloud SQL or Cloud Spanner | Cloud SQL for regional transactions; Spanner for global consistency |
| Real-time operations | Bigtable or BigQuery | Bigtable for low-latency operations; BigQuery for large SQL analysis |
| Mobile backend | Firestore | Document model with live sync and offline support |
Exam tip: Choose Cloud Spanner when relational transactions need global consistency and horizontal scale; choose Cloud SQL when a regional managed database is enough.
Object storage manages data as distinct, immutable units called objects inside containers called buckets. Cloud Storage (opens in a new tab) is the scalable managed object service where each bucket can hold essentially unlimited objects of any format. Each object bundles file data with metadata and a unique identifier, and buckets define storage class and location that control durability and cost.
The resource hierarchy flows from organization to project to bucket to object, with names such as projects/_/buckets/BUCKET_NAME for buckets. Folders in hierarchical-namespace buckets provide a real file system where renames are atomic, while managed folders are permission overlays that add access beyond bucket permissions. Buckets also serve as a foundation for analytics and AI, because services such as Vertex AI and Google Kubernetes Engine (GKE) can read directly from object storage.
| Bucket location type | Replication scope | Replication behavior |
|---|---|---|
| Regional | Zones in one region | Data replicated synchronously across zones |
| Dual-region | Two geographic locations | Metadata synchronous; data asynchronous; turbo replication with 15-minute RPO |
| Multi-region | Two or more locations | Metadata synchronous; data asynchronous |
Exam tip: Object storage keeps immutable objects in buckets, while block storage provides network disks that virtual machines use like physical disks.
A relational database stores highly structured data in tables of rows and columns with links across tables. Each table holds one kind of record, each row is one record, and each column is one data point, so related data connects without duplication. A relational database management system maintains those links and can use indexes for faster lookups.
Structured Query Language (SQL) (opens in a new tab) is the language for storing, retrieving, and managing relational data with statements such as CREATE, INSERT, and SELECT. In BigQuery, GoogleSQL groups work into query statements for analysis, Data Definition Language for creating objects, Data Manipulation Language for updating rows, Data Control Language for access and capacity, Transaction Control Language for transactions, and procedural statements for multi-statement logic. On Google Cloud, Cloud SQL is the fully managed relational service that handles setup, patches, backups, and replication.
| Engine | Ownership and type | Key facts |
|---|---|---|
| MySQL | Open-source relational | Supports SQL by interface or command line; deployable manually or managed |
| PostgreSQL | Open-source object-relational | Valued for performance and robust features with many procedural extensions |
| SQL Server | Microsoft-owned relational | Supports transactions, analytics, and machine learning with row-based tables |
Non-relational databases, also called NoSQL databases, store data without forcing every record into fixed tables. These services increase flexibility for changing fields and scale automatically as volume grows. Google Cloud labels them by model, and each label signals the shape it handles best.
Firestore (opens in a new tab) is the serverless document database for mobile, web, and server apps, including Firestore with MongoDB compatibility for existing MongoDB apps. Datastore is a second document database for web and mobile apps needing automatic scaling. Bigtable is the wide-column and key-value service for terabyte to petabyte analytical and operational workloads accessed by key. Memorystore holds data in memory with Valkey or Redis to reduce latency, while Memorystore for Memcached is deprecated for new designs.
| Service | Data model | Best fit |
|---|---|---|
| Bigtable | Wide-column and key-value | Terabytes to petabytes with key-driven access |
| Firestore | Document | Rich mobile and web apps with flexible fields |
| Datastore | Indexed document | Web and mobile apps needing automatic scaling |
| Memorystore | In-memory | Extreme performance with Valkey or Redis |
Exam tip: Bigtable is the wide-column, petabyte-scale NoSQL service, while Firestore and Datastore are document databases for application development.
Advanced analytics and integrated machine learning in BigQuery let high-speed analysis and modeling happen together in one warehouse. BigQuery (opens in a new tab) is a fully managed, AI-ready platform that can query terabytes in seconds and petabytes in minutes with no infrastructure to manage. Analysts use familiar SQL and Python for business intelligence, ad hoc exploration, geospatial analysis, and machine learning where the data already lives.
A query flows from the user to the serverless compute layer, the compute pulls data from separate storage over a petabit-scale network, and results flow back without provisioning servers. GoogleSQL supports joins, nested fields, aggregations, and spatial functions, and federated queries can run against Cloud Storage, Bigtable, Spanner, or Google Sheets without loading first. BigQuery ML extends this by storing models in datasets like tables, so training data never leaves the warehouse for a separate framework.
Exam tip: Choose BigQuery ML when the model should be trained and run with SQL where the warehouse data already lives.
Multicloud analysis with BigQuery Omni analyzes data in other clouds through one BigQuery interface without moving data. BigQuery Omni (opens in a new tab) runs compute locally to data stored in Amazon Simple Storage Service (Amazon S3) or Azure Blob Storage. Because data stays in place, organizations avoid movement and egress costs while reducing silos.
The workflow follows a fixed order. First connect BigQuery to Amazon S3 or Azure Blob Storage, then create a BigLake table that references the external data in place. Then run analytics in a BigQuery Omni region near that storage, and optionally join results with local BigQuery tables or move results into BigQuery. Compute runs near external storage, so cross-cloud joins between AWS and Azure regions are possible without first copying everything.
| Tool | What it does | Suggested use |
|---|---|---|
| Omni joins | Joins Omni-region data from a BigQuery region | One-time queries without caching |
| Omni materialized views | Replicates Omni data incrementally | Repeated queries where caching matters |
| Transfer using SELECT | Moves query results to a BigQuery region | Bringing results into BigQuery |
| Transfer using LOAD | Loads directly from S3 or Blob Storage | Loading external data into BigQuery |
Serverless design in BigQuery means teams focus on data instead of servers, scaling, and maintenance. There is no provisioning of servers or storage systems, no manual scaling for larger queries, and no downtime-driven maintenance, because Google handles updates behind the scenes. A team can start querying in minutes and grow to very large analysis without redesigning infrastructure.
Separation of storage and compute is what makes this scaling work. The storage layer ingests and optimizes data in a columnar format replicated across locations, while the compute layer provides distributed query power. The two layers communicate over a high-speed network and allocate resources independently, so storage and compute improvements deploy without downtime.
| System | How resources work | Effect on performance |
|---|---|---|
| Legacy databases | Reads, writes, and analytics share one pool | Queries slow during writes |
| BigQuery | Storage and compute scale independently | Queries and storage grow without blocking |
Access frequency determines storage class by matching how often objects are read to the right cost profile. Cloud Storage (opens in a new tab) keeps data as objects in buckets, where each bucket and object has a class and location that set durability and cost. Standard is the default for active data, Nearline and Coldline are for rarely accessed data, and Archive is the cheapest and most durable class for legal and long-term retention.
Placement follows retrieval needs, so data that stops being read daily can move colder. Storage Intelligence helps decide with real use data: the advisor finds idle buckets and unoptimized classes, Insights datasets expose metadata for BigQuery analysis, Gemini Cloud Assist answers natural-language questions, and inventory reports catalog objects per bucket. Storage batch operations can then change classes across many objects at once.
| Storage class | Access pattern | Cost note |
|---|---|---|
| Standard | Frequent access | Default class with no special discount |
| Nearline | Rarely accessed | Lower storage billing than Standard |
| Coldline | Rarely accessed | Lower storage billing than Standard |
| Archive | Legal and long-term retention | Cheapest and most durable |
Exam tip: Standard is the default class. Move rarely accessed data to Nearline or Coldline to save on billing, and use Archive for data kept for legal reasons.
Storage cost versus retrieval cost is the central tradeoff where colder storage costs less to keep but more to read. Each class has its own pricing for storing data and for reading data or metadata back, so changing class changes both sides of the bill. Archive is economical only when reads are rare, because any retrieval costs more than in a warmer class. Nearline fits backup scenarios with access at most about once a month.
Minimum durations add a second rule for Nearline, Coldline, and Archive. The account is charged for the minimum duration even if the object is deleted or moved earlier, plus extra costs for early retrieval of data or metadata. Short-lived data can therefore cost more in a cold class than in Standard, even though the monthly storage rate looks lower. Effective planning considers how long objects will stay, not just the storage rate.
| Storage class | Relative storage cost | Relative retrieval cost | Typical access |
|---|---|---|---|
| Standard | Highest | Lowest | High-frequency active objects |
| Nearline | Lower than Standard | Higher than Standard | About once a month |
| Coldline | Lower than Nearline | Higher than Nearline | Lower than Nearline |
| Archive | Cheapest | Highest | Lowest-frequency preservation |
Use cases for each class connect business needs for access, cost, and durability to the right choice. Standard (opens in a new tab) fits active content delivery and web serving with high-frequency access. Nearline fits backup and disaster recovery with regular stress tests, Coldline fits rarely accessed data, and Archive fits archival and legal retention kept for years but almost never read.
Protection and import services support these plans. Soft Delete preserves deleted objects for a set period against accidents, Object holds block deletion or updates, and Storage Transfer Service imports existing backups from Amazon S3, Azure Blob Storage, or on-premises sources. Lifecycle planning then moves data colder as access drops, but must watch retrieval charges and minimum durations so savings are real.
Migration strategies describe how much change an organization makes when moving a database to Google Cloud. The three main choices are lift-and-shift with no schema change, replatforming to a managed service, and modernizing to a cloud-native design. Managed services improve long-term operation because replication, backup, and elastic scaling are built in.
Lift-and-shift moves schema and data as-is through staging files and Cloud Storage (opens in a new tab) into BigQuery, so old pipelines keep working while proofs of concept explore cloud features. Replatforming moves to Cloud SQL, Bigtable, or Firestore with little application change while gaining managed replication and durability. Modernization redesigns for native features, such as evolving BigQuery star schemas to nested and repeated fields with partitioning and clustering to scan less data.
| Strategy | Change required | Choose when |
|---|---|---|
| Lift-and-shift | Move as-is | Speed and continuity matter most |
| Replatform | Move to managed service | Reliability is needed without rewriting |
| Modernize | Redesign schema | Scale and long-term efficiency justify effort |
Exam tip: Transfer the schema unchanged first to keep pipelines running, then evolve to nested fields and clustering; AVRO, ORC, and Parquet embed schema, while CSV needs a separate definition.
Migration tools move bulk files, live databases, and server images with less downtime. Storage Transfer Service (opens in a new tab) brings data to Cloud Storage from Amazon S3, Azure Blob, data lakes, or on-premises systems. Transfer Appliance is a physical appliance filled on-site and shipped to Google for very large moves, while BigQuery Data Transfer Service automates scheduled moves into BigQuery from software-as-a-service apps, Teradata, and Redshift.
| Tool | Source to destination | Choose when |
|---|---|---|
| Storage Transfer Service | Clouds and file systems to Cloud Storage | Data can move over the network |
| Transfer Appliance | On-premises to Google Cloud | Data exceeds 20 TB, up to 1 petabyte |
| BigQuery Data Transfer Service | SaaS and warehouses to BigQuery | Goal is scheduled refactor to BigQuery |
Database migration jobs move engines such as Amazon RDS to Cloud SQL with either planned-downtime restore or continuous replication followed by cutover. Specialized paths include HarbourBridge for Spanner evaluation, HBase tools for Bigtable, and Hive-to-BigQuery frameworks that track progress in Cloud SQL. Server moves use Image Import for virtual disks into Compute Engine images, or RackWare and SUREedge from Marketplace when other tools do not fit. After data lands, Cloud Build creates artifacts, Artifact Registry stores images, and Cloud Deploy promotes releases to targets such as GKE.
Modernization paths define how each database moves and what limits apply before cutover. Database Migration Service (opens in a new tab) migrates SQL databases to Cloud SQL, but a Cloud SQL to Cloud SQL move can go to a different region, not a different project. Replication creates a replica in a new region for Cloud SQL and Bigtable, while backup and restore moves Filestore, Bigtable, Firestore, and Cloud SQL images. Export and import through Cloud Storage moves Bigtable, Firestore, and Spanner data, with Spanner using Dataflow templates to Avro files.
Refactoring to BigQuery decouples storage from compute using slots, with a default limit of 2,000 slots and long-term storage dropping price by 50 percent after 90 days without edits. Firestore regional moves use export and import through a bucket, but backup and restore is less expensive and more straightforward, and locations may tie to App Engine presence. Only shut down the source after extended tests succeed, keep backups temporarily, and consider keeping the old region as a disaster recovery site.
| Service | Type | Pathway and consideration |
|---|---|---|
| BigQuery | Warehouse | Transfer, ETL, and denormalization; slots and 90-day discount |
| Firestore | Document | Export/import or backup/restore; backup/restore is cheaper |
| Spanner | Relational | Dataflow export to Avro and re-import |
Choose Cloud SQL or Cloud Spanner for operational transactions on current records, BigQuery for analytical queries over large datasets, and Cloud Storage for objects such as files, images, backups, and static website assets. Cloud SQL provides regional MySQL, PostgreSQL, and SQL Server for row-level transactions, while Cloud Spanner provides global-scale relational transactions that stay available across regions.
Cloud Firestore is a document NoSQL database with schema flexibility, real-time sync, offline support, and automatic scaling for mobile, web, and IoT apps, while Cloud Bigtable is a wide-column NoSQL database for massive operational throughput with high throughput and low latency. Cloud Bigtable does not support complex joins or multi-row transactions, whereas Firestore stores documents in collections for use cases such as user profiles and catalogs.
Standard is the default class for frequent access, Nearline and Coldline are for rarely accessed data, and Archive is the cheapest and most durable class for legal and long-term retention. Colder classes cost less to store but more to retrieve, and Nearline, Coldline, and Archive charge for a minimum duration even if an object is deleted or moved earlier.
BigQuery Omni analyzes data stored in Amazon Simple Storage Service (Amazon S3) or Azure Blob Storage through one BigQuery interface without moving the data. You connect BigQuery to the external storage, create a BigLake table that references the data in place, and run analytics in a BigQuery Omni region near that storage, which avoids movement and egress costs.
Last updated