Prepare and test your skills

Prepare and test your skills

Managed export and import is a way to copy Firestore (opens in a new tab) data through Cloud Storage for recovery, migration, and offline use. Data flows from the source database to export files in a Cloud Storage bucket, and then from those files into the same database or a different database during import. This flow supports recovery from accidental deletion, offline processing, loading exports into BigQuery, and moving data between projects.
The tools for Firestore export and import are the Google Cloud console, the gcloud command-line tool, and the Admin APIs over REST and RPC. An operator starts an export that writes database contents to a Cloud Storage path, and later starts an import that reads from that path back into a database. Data exported from one Firestore with MongoDB compatibility database can be imported into another database of the same type, and data exported in Datastore mode (opens in a new tab) can be imported into another Datastore mode database, even in a different project.
Prerequisites for export and import ensure billing, storage location, and permissions are ready before any data moves. The database project must have billing enabled, and a Cloud Storage bucket must already exist near the database location. The operation also depends on the Firestore service agent to authorize Cloud Storage access.
Owner, Cloud Datastore Owner, or Cloud Datastore Import Export Admin, plus Owner or Storage Admin for Cloud Storage access.gcloud: Configure gcloud for the correct project with gcloud config set project project-id, either in Cloud Shell or with an installed Google Cloud CLI.If the bucket is in another project, the source or destination project service account must be granted access to that bucket. The Firestore service agent uses the name service-PROJECT_NUMBER@gcp-sa-firestore.iam.gserviceaccount.com.
A Firestore export captures documents or entities, with options for a full or filtered copy. You can export all documents or only specific collections, and you can export all entities or only specific kinds or namespaces in Datastore mode. Likewise, you can import all data from an export or only specific collections or kinds.
Exam tip: Keep the export metadata filename identical to its parent folder, as gs://BUCKET_NAME/OPTIONAL_NAMESPACE_PATH/PARENT_FOLDER_NAME/PARENT_FOLDER_NAME.overall_export_metadata, or the import cannot find the files.
The output format for a Datastore mode managed export uses the LevelDB log format. The export writes data files plus metadata files into subfolders under a parent folder in Cloud Storage. To import only a subset of entities or to import data into BigQuery, the export must specify an entity filter. If export output files are moved or copied, the parent folder name, subfolder contents, and .overall_export_metadata filename must stay the same.
Moving Firestore data between projects (opens in a new tab) is an export-to-bucket followed by an import-from-bucket across two projects. The source project writes the data files, the destination project is granted read access, and the destination project loads the data.
This cross-project flow depends on matching preparation on both sides. Both source and destination projects need billing enabled and accounts with export and import roles, and composite indexes should be set up in the destination project first to avoid processing each document multiple times.
Long-running operations are the export and import jobs that continue in the background until they complete. Operators can list all export and import operations, check the status of a single operation, cancel a running operation, and delete operation records. For Datastore mode, completion time can be estimated while the operation runs, and billing and pricing details for managed exports and imports can be viewed separately from normal usage.
Automated and manual backup mechanisms protect Cloud SQL (opens in a new tab) instances and AlloyDB for PostgreSQL clusters by saving recovery points to protected storage. Automated backups run on a schedule defined in a backup plan, while manual backups, called on-demand backups, are triggered by a person at any time. Both types use the same Backup and DR (opens in a new tab) vault storage and retention controls, but they differ in how they start and how long they are kept.
Automated backup schedules are defined by a backup plan that is assigned to a Cloud SQL instance or AlloyDB cluster. Once the plan is assigned, Backup and DR automatically captures and retains backups according to the backup rules in the plan, with each backup starting inside a defined backup window. The backup window uses a 24-hour clock format and must be at least six hours long, so it is best to schedule backups when the instance has less activity if possible.
Each backup rule in a backup plan specifies how and where automatic protection happens:
Hourly frequency limits depend on the resource type. A Cloud SQL instance supports an hourly frequency of 6 to 23 hours, while an AlloyDB cluster supports 1 to 23 hours. The first scheduled backup is automatically triggered when the current time is within the backup window.
Manual on-demand backups are immediate backups triggered by a person from the Vaulted backups page by selecting a resource and choosing Create on-demand backup. An on-demand backup for Cloud SQL can be created by triggering an existing backup rule immediately or by specifying a user-defined custom retention period. An on-demand backup for AlloyDB is created by triggering the backup rule to run immediately.
| Backup type | How it starts | Choose when |
|---|---|---|
| Automated backup | Starts automatically per the backup plan rule during the backup window | For regular, ongoing protection of Cloud SQL and AlloyDB |
| On-demand backup | Starts manually from Vaulted backups using Create on-demand backup | Before a risky operation or when you need a backup without waiting for the window |
On-demand backups are ideal for capturing a known good recovery point before major changes or for ad hoc protection. You can create a Cloud SQL on-demand backup (opens in a new tab) for any instance, whether automatic backups are enabled or not. The on-demand backup is stored in the backup vault that is configured with the associated backup plan.
Backup retention policies control how long a backup is kept before it expires and whether it can be deleted early. Every backup is retained for the period assigned in the backup plan and then expires, but backups stored in a backup vault are immutable and cannot be modified or deleted during the vault minimum enforced retention period. You can optionally set a Prevent deletion period on the vault that is longer than the Delete backups after setting, and the backup will not be deleted until the enforced period ends.
For automated backups, retention can range from 1 day to 10 years depending on the backup option and plan configuration. For on-demand backups, standard backups are stored indefinitely until manually deleted, while enhanced backups use a custom retention period up to 10 years as configured in the associated backup plan. Backup retention periods cannot be less than the backup vault minimum retention period and must be equal to or greater than it.
Exam tip: Standard on-demand backups are kept indefinitely until deleted, while enhanced on-demand backups follow the retention rule in the backup plan up to 10 years.
Backup storage locations are centered on a backup vault, which is a centralized storage container where backup sets are stored. A backup vault provides immutability and enforced retention to protect against accidental or malicious deletion. You can configure a vault to store backups in a single region or across multiple regions.
The backup vault and backup plan must be in the same project and in compatible locations. Storage location also depends on the Cloud SQL backup option. Standard backups are created, managed, and stored in the same project as the Cloud SQL instances, while enhanced backups are managed and stored in a centralized backup management project that leverages the Backup and DR Service. Enhanced backups provide enforced retention, granular scheduling, and monitoring, with extra protection for customer-managed encryption key instances because key access is granted to the Backup and DR service agent in the vault project.
Point-in-time recovery (PITR) is the ability to restore a database to its state at a specific timestamp, not just to the time of the last full backup. Cloud SQL (opens in a new tab) combines incremental backups with transaction logs to support PITR, while Firestore uses scheduled consistent backups to support restore to a prior point. Both approaches reduce data loss by capturing changes between backups and keeping restorable copies until retention expires.
How point-in-time recovery works is through two layers of protection that combine to reach a specific moment. A backup provides the base copy of the data at a known time, and transaction logs record every write that happened after that backup. During recovery, the database system restores the base backup and then replays the transaction logs forward to the requested timestamp. This combination allows recovery to a point before an error, such as accidental deletion or corruption, with minimal data loss.
Cloud SQL point-in-time recovery recovers an instance to a specific point in time by applying transaction logs, also called binary logs, on top of a backup. A point-in-time recovery in Cloud SQL always creates a new instance, and the recovery cannot be performed to an existing instance. Because the new instance starts from stored configuration and replayed data, the original instance remains unchanged while the recovered copy can be verified before switching traffic.
A horizontal timeline showing a Cloud SQL backup as base copy, continuous transaction logs, a PITR timestamp before an error, and a replay arrow creating a new instance.
Successful Cloud SQL recovery depends on retaining both backups and logs:
Cloud SQL backups are incremental and encrypted by default with Google-managed keys or customer-managed encryption keys (CMEK). All backups except the first contain only data that changed after the previous backup, and the oldest backup is similar in size to the database. High write activity can generate a large volume of transaction logs, which consumes disk space and can cause automatic storage growth, so instance storage must be sized to account for transaction log retention.
Exam tip: A Cloud SQL backup restores to the time the backup was taken, while point-in-time recovery restores to a specific timestamp between backups and always creates a new instance.
Cloud SQL data protection uses backups, point-in-time recovery, and exports together because each protects against a different scenario. Backups are lightweight and managed by retention policies, point-in-time recovery adds timestamp precision, and exports create portable files in Cloud Storage. The following comparison shows when each option fits.
| Option | Recovery point | Restore behavior |
|---|---|---|
| Cloud SQL backup | State at the time the backup was taken | Restores the entire disk of the instance; cannot back up a single database or table, and cannot restore from the backup if the source region is unavailable |
| Point-in-time recovery | Specific timestamp before an error occurred | Always creates a new instance; cannot recover to an existing instance |
| Export | State captured in an external file in Cloud Storage | Takes longer to create, but survives instance deletion and can export only a single database or table depending on the format |
Backup and restore operations cannot be used to upgrade to a later database version. A restore works only to an instance with the same database version as when the backup was taken. Backups are not available for replica instances because backups are maintained with the primary instance, and a promoted replica needs its own backup configuration and cannot access the former primary backups.
Firestore scheduled backups provide point-in-time copies that can be used to restore data to a new database after corruption or accidental deletion. A Firestore backup is a consistent copy of the database at a point in time and contains all data and index configurations at that time. A Firestore backup does not contain database time-to-live policies, resides in the same location as the source database, and creating or retaining application-consistent backups (opens in a new tab) does not affect read or write performance on the live database in the same way Firestore backups leave live performance unaffected.
Firestore backup schedules control how often those recovery points are created. Each database can have up to one daily backup schedule and up to one weekly backup schedule, and multiple weekly schedules for different days are not supported. The exact time of day cannot be configured, and backups are taken at different times each day, although a weekly schedule can specify the day of the week. Backups have a configurable retention period and are stored until the period expires or the backup is deleted, and deleting the source database does not automatically delete related backups.
Automated backups start automatically according to the backup plan rules during the defined backup window for regular, ongoing protection, while on-demand backups are triggered manually from the Vaulted backups page by selecting a resource and choosing Create on-demand backup. Both types use the same Backup and DR vault storage and retention controls, but automated backups follow the plan schedule and on-demand backups capture an immediate recovery point before a risky operation or for ad hoc protection.
A Cloud SQL backup restores the entire disk of the instance to the state at the time the backup was taken, while Point-in-Time Recovery (PITR) restores to a specific timestamp by replaying transaction logs on top of a backup. Point-in-Time Recovery (PITR) always creates a new instance and cannot recover to an existing instance.
The database project must have billing enabled, with Firebase projects on the Blaze plan, and a Cloud Storage bucket must already exist in the same location as or near the Firestore database location. The account starting the operation needs export and import permissions such as Owner, Cloud Datastore Owner, or Cloud Datastore Import Export Admin plus Owner or Storage Admin for Cloud Storage access, and the operation depends on the Firestore service agent to authorize Cloud Storage access.
An operations team manages a production Cloud SQL instance supporting a critical web service. At 2:15 PM, an operator accidentally runs a query that corrupts table data. The team needs to restore the database state to 2:14 PM—one minute prior to the corruption—with minimal data loss.
Automated daily backups run every midnight.
Which strategy should the team implement to restore the database to the specified timestamp?
Last updated