Diagram

A sequence diagram showing the automated BigQuery backup workflow: Cloud Scheduler triggers Pub/Sub, the dispatcher lists tables in scope, the configurator applies table-level or fallback policies, a snapshot or export service runs the backup via the BigQuery API, and Cloud Logging triggers the tagger to record backup state in Cloud Storage metadata. Arrows appear in order to show the message flow.
Automated BigQuery Backups at Scale
Automated BigQuery backup uses scheduled services to create recurrent copies of tables across folders, projects, datasets, and tables. Each table is backed up by one of two methods: a BigQuery snapshot, which is a point-in-time table copy inside BigQuery, or a BigQuery export to Cloud Storage, which writes table data as files in Cloud Storage. Each table gets its backup policy from a table-level policy defined by data owners, or from a fallback policy defined by a data governance officer when no table-level policy exists. This design retries through Pub/Sub when BigQuery concurrency limits cause failures, and it spreads snapshot or export operations across projects or requests a quota increase when daily limits are reached.
Domain 5 of 7
22 practice questions
Practice this DomainLast updated