Choosing Cloud Storage and Database Solutions

Choosing the appropriate data storage location type means selecting where Cloud Storage (opens in a new tab) bucket data physically lives and how it is protected. The location type is chosen when the bucket is created and is permanent, so geographic spread, redundancy, and cost must be balanced up front. The four types are zonal, regional, dual-regional, and multi-regional, and regions are independent geographic areas designed as independent failure domains while zones are deployment areas inside a region that act as single failure domains.

1.1Availability Protection and Failover by Location Type

Availability protection by location type describes whether Cloud Storage data remains accessible when a zone or a region has an outage. Zonal storage keeps redundant copies inside one availability zone, so a zonal outage can affect availability. Regional storage replicates data synchronously across availability zones in the same region, while dual-regional and multi-regional storage replicate data asynchronously across geographically separated regions for 99.99% availability.

Regional, dual-regional, and multi-regional buckets provide automated failover and failback with recovery time objective (RTO) = 0 and no change to storage paths because Google Cloud manages redirection automatically. Dual-regional buckets offer higher availability than regional buckets for the same storage class and support Turbo replication, which replicates 100% of newly written objects between the paired regions within 15 minutes. Multi-regional buckets also provide higher availability than regional buckets for the same storage class through cross-region replication for users spread over a wide area.

1.2Latency and Placement of Storage Near Compute

Latency and placement of storage near compute describes the network delay between the application that reads or writes data and the location where that data is stored. Colocating application components within the same region generally minimizes network latency between components, which can improve application performance and user experience. A zonal bucket offers the highest performance in Cloud Storage for demanding workloads because compute and storage are closest together inside one zone.

Regional storage provides low latency when the application runs in the same region, and reads inside the same region avoid cross-region network hops. Dual-regional storage allows reads from within either of its two regions without leaving that local path, which helps applications that operate in two regions. Protection against a full regional outage uses asynchronous replication across regions, so it comes with replication delay rather than the synchronous zone-to-zone copy used inside a region.

1.3Storage Price, Replication Charges, and Transfer Cost by Location

Storage price by location type varies across storage price, replication charges, and data transfer charges for reads. Regional storage is the lowest-cost option for data that can stay in one region, while dual-regional and multi-regional storage add replication cost to keep copies in more than one region. Data flows from the writer to replicated copies on every write in dual-regional and multi-regional buckets, which adds replication charges that a regional bucket avoids.

Matching location type to application needs means choosing the smallest failure scope that still meets uptime goals and the closest placement that still meets speed goals. An application that must keep running through a regional outage needs cross-region redundancy, while an application that only needs to survive a zonal outage can use regional protection.

• Choose zonal when the workload needs the highest storage performance in one zone and can tolerate reduced availability during a zonal outage.
• Choose regional when the workload runs in one region, needs protection against a zonal failure with automatic failover, and benefits from low latency to nearby compute.
• Choose dual-regional when the workload runs in two regions, needs protection against a regional failure, and needs fast replication between the pair with Turbo replication.
• Choose multi-regional when the workload needs protection against a regional failure and wide geographic availability beyond a single pair of regions.

1.4Data Residency Controls and Compliance Boundaries for Locations

Data residency controls and compliance boundaries determine the physical location of data plus the local regulations that govern how that data is stored, encrypted, and accessed. Administrators must know the physical location of data centers to research which data locality laws apply, because personally identifiable information (PII) must be stored and processed according to the rules of that region. Examples include a country that mandates government-related data stay inside that country, or a customer contract that requires data for certain users to stay in a different country.

Cloud SQL (opens in a new tab) supports compliance by letting the instance region control where data rests and who can reach it. The region is selected when the Cloud SQL instance is created and can be changed by editing an existing instance, and Google stores data at rest only in the configured regions, including backups. A cross-region read replica places a readable copy in a designated region to help meet residency standards for that region, while network controls limit the locations where users can access data and limit administrator access. Organizational policy constraints enforce these choices at the organization, folder, or project level by defining the Google Cloud locations where users can create resources for supported services, so new resources cannot be placed outside allowed boundaries.

Classifying use cases into structured, unstructured, or semi-structured data requirements means checking how fixed the data shape is and then choosing the Google Cloud service that fits that shape. Structured data has a fixed schema and fits tables, unstructured data has no fixed schema and fits files and objects, and semi-structured data has a flexible or nested shape and fits document and wide-column stores. In Google Cloud, BigQuery serves structured analytics, Cloud Storage (opens in a new tab) serves unstructured objects, and Firestore in Datastore mode and Bigtable serve flexible large-scale workloads.

2.1Structured Data Traits and SQL Storage Options

Structured data is data organized into fixed fields with known names and types, such as tables with rows and columns. Each record holds the same kind of attributes, for example an id, a title, a rating, and an available_date, so values can be filtered, joined, and aggregated reliably. Because the layout is predictable, structured data fits a fixed schema and SQL-compatible querying in Cloud SQL and BigQuery.

Fixed schema means structured data defines field names, data types, and required values before the data is stored and queried. In BigQuery, a table schema can be auto-detected on import or supplied as a JSON object through the API, and editing the auto-detected schema to map key properties such as titles is recommended. Structured records stored in Cloud Storage must use JSON Lines or NDJSON format, where each line is one document made of a set of fields, and each file must be less than 2 GB, each row must be less than 1 MB, and up to 1,000 files can be imported in a single request.

SQL compatibility means structured data can be stored in tables and queried by column values, primary keys, and relational operations. Cloud SQL provides a relational database with full SQL support, where a table holds rows, each row holds columns, and a primary key gives each row a unique ID. BigQuery provides interactive querying for large tabular datasets, including public datasets that follow the same table-and-column pattern. The decision depends on workload type and query pattern:

Firestore in Datastore mode (Datastore) (opens in a new tab) shows the same structured idea in NoSQL form with a SQL-like query language. Its concepts map directly to relational concepts where kind maps to table, entity maps to row, property maps to column, and key maps to primary key. Unlike a relational database, Datastore is schemaless and does not require every entity of the same kind to have the same properties, and its queries scale with the size of the result set rather than the size of the dataset. Datastore does not support joins, inequality filtering on multiple properties, or filtering based on subquery results.

2.2Unstructured Data Traits and Object Storage Behavior

Unstructured data is content without a fixed schema, where each file is stored and read as a whole object rather than as queryable rows. Common examples include multimedia and archives such as PDFs, scanned images, videos, large images, movies, website assets, backups, and regulatory archives. Cloud Storage is the object storage service for unstructured data, and it stores data as objects inside a flat hierarchy of buckets where each object has a globally unique ID plus system-assigned and user-defined metadata.

Applications access objects by ID using REST APIs or client libraries, and data can be accessed from inside and outside Google Cloud. Data flows from a source bucket or folder into processing, and import is recursive, so files inside subfolders are included when a bucket or folder is selected. Typical searchable formats include TXT, PDF, and HTML for documents, DOCX, PPTX, XLSX, and XLSM for office files, and JSON, Markdown, XHTML, and XML when stored as text, with limits of less than 200 MB per file and up to 100,000 files at a time.

When unstructured documents need descriptive fields, the unstructured files stay in Cloud Storage while a BigQuery table or NDJSON file holds the metadata, and the uri field in each metadata record points to the Cloud Storage location plus its mimeType, such as application/pdf. Unstructured import with metadata follows an ordered flow:

1. Create metadata: Create a BigQuery table or NDJSON file with id, jsonData or structData, and content fields.
2. Store documents: Put the unstructured documents into a Cloud Storage bucket, which can be the same bucket as the metadata or a different bucket.
3. Link by URI: Data flows from the metadata record to the document because the uri field in each record points to the Cloud Storage location of that document.

2.3Semi-Structured Data Traits and NoSQL Storage Options

Semi-structured data has an organizing structure such as fields, keys, and nested objects, but it does not require every record to share the same schema. Examples include NoSQL documents with nested fields like hotel records with titles, locations, ratings, and room-type arrays, plus user profiles, product catalogs, and wide-column records where columns can vary by row. Firestore in Datastore mode is the NoSQL document database for semi-structured application data that needs automatic scaling, high availability, and strong consistency.

Datastore organizes data as kinds containing entities, where each entity has a key and a flexible set of properties, so entities of the same kind can have different properties or different value types for the same property name. Datastore provides atomic transactions where all operations succeed or none occur, high availability through redundancy, strong consistency for queries, encryption at rest, and fully managed operation with no planned downtime. Reads scale because query performance scales with the size of the result set rather than the size of the dataset, so a query returning 100 entities performs the same whether it searches hundreds or millions of entities.

Bigtable is the wide-column NoSQL option for data that is not highly structured and does not require ACID transactions, especially at very large scale with high read and write throughput. For workloads that do not require ACID transactions or whose data is not highly structured, Bigtable is the alternative to Datastore. Matching data shape to service therefore depends on schema flexibility, consistency needs, and scale:

2.4Access Frequency and Retention Classes for Object Storage

Access frequency and retention classes further refine the Cloud Storage choice because each storage class trades retrieval cost for retention length. Frequently accessed data such as data lakes, websites, streaming video, and mobile content uses Standard, while predictable infrequent data uses longer-retention classes. When access patterns vary or retention is unpredictable, Autoclass automatically transitions each object between classes based on its access pattern.

• Standard for frequently accessed data.
• Nearline for infrequent access with at least 30 days retention.
• Coldline for infrequent access with at least 90 days retention.
• Archive for infrequent access with at least 365 days retention.
• Autoclass for automatic transitions when access is unpredictable.

Key Facts

• Cloud Storage bucket location is chosen at creation, is permanent, and can be zonal, regional, dual-regional, or multi-regional.
• Regional storage replicates synchronously across zones while dual-regional and multi-regional storage replicate asynchronously across regions, all with automatic failover and RTO = 0.
• Turbo replication replicates 100% of newly written objects between dual-regional paired regions within 15 minutes.
• Regional storage has the lowest storage price with no replication charges and no outbound charges for same-region reads, while multi-regional reads always incur outbound charges.
• Structured data uses a fixed schema queried with SQL in Cloud SQL for OLTP and BigQuery for OLAP, unstructured data is stored as Cloud Storage objects accessed by ID, and semi-structured data uses Firestore in Datastore mode for flexible ACID documents or Bigtable for wide-column scale without ACID.