Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Google Cloud feature
partitioned tables
Topic in 7 exam domains
- partitioned tables · Feature
- partitioned tables · Concept
Also called: Partitioning, partitions, time-unit partitioned tables, TimePartitioning
Tables divided into segments based on defined column characteristics, such as time units or integers, reducing bytes read by queries.
Where this appears on the exam
5 objectives in 3 of 5 domains
Related topics
FoundationsBuilding blocksBuilding blocksThis topicVPC Service ControlsServiceCloud StorageServiceDatastreamServiceGoogle Cloud StorageServiceKnowledge CatalogServiceBigQueryServiceCloud SQLServiceDataflowService
partitioned tablesFeature
clustered tablesFeaturePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- partitioned tables is part of BigQuery From the official documentation: “There are two types of partitioned tables in BigQuery:”
- clustered tables works with partitioned tables From the official documentation: “You can combine table clustering with [table partitioning](/bigquery/docs/partitioned-tables) to achieve finely-grained sorting for further query optimization.”
- clustered tables is an alternative to partitioned tables From the official documentation: “If you exceed<br>this limit, consider using [clustering](/bigquery/docs/clustered-tables) in addition to, or instead of, partitioning.”
- Datastream works with partitioned tables From the official documentation: “Datastream supports replicating partitioned tables for PostgreSQL source databases.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
- BigQuery works with Datastream From the official documentation: “Permissions needed for datastream to write to BigQuery.”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
- Cloud Storage works with Datastream From the official documentation: “When you stream to Cloud Storage, Datastream organizes data by object name and timestamp.”
- VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
- BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
- Cloud SQL works with Cloud Storage From the official documentation: “When ingesting data from Cloud SQL, data is first staged to a Cloud Storage bucket.”
- BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
- clustered tables is part of BigQuery From the official documentation: “BigQuery supports clustering for both partitioned and non-partitioned tables.”
- BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
- Cloud SQL works with Knowledge Catalog From the official documentation: “You can use Knowledge Catalog to organize and enrich the metadata of Cloud SQL resources.”
- Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
- VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
- Cloud SQL works with Datastream From the official documentation: “This page describes an example NAT VM configuration that lets Datastream privately connect to a Cloud SQL instance.”
- Cloud SQL works with Google Cloud Storage From the official documentation: “Imports data into a Cloud SQL instance from a SQL dump or CSV file in Cloud Storage.”
- VPC Service Controls secures Cloud SQL From the official documentation: “VPC Service Controls perimeters protect the Cloud SQL Admin API.”
- Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
- Dataflow works with Datastream From the official documentation: “In addition, Datastream supports writing the change event stream into Cloud Storage, and offers streamlined integration with Dataflow templates to build custom workflows for loading data into a wide range of destinations, such as Cloud SQL and Spanner.”
- Dataflow depends on Google Cloud Storage From the official documentation: “- A gcs bucket for staging DataFlow files. Replace `<BUCKET_NAME>`: the name of a valid Google Cloud Storage bucket.”
- Dataflow works with Google Cloud Storage From the official documentation: “Data is transferred between Spanner and Cloud Storage using Dataflow.”
- Datastream works with Google Cloud Storage From the official documentation: “| How are files created in Cloud Storage? | Datastream creates a folder for each table.”
- Datastream works with Knowledge Catalog From the official documentation: “You can search for and manage your Datastream resources using Knowledge Catalog, which is a platform for storing, managing, and accessing your metadata.”
- VPC Service Controls secures Google Cloud Storage From the official documentation: “Storage Transfer Service supports on-premises transfers to Cloud Storage buckets protected by VPC Service Controls, under the following conditions:”
- VPC Service Controls secures Knowledge Catalog From the official documentation: “The API for Knowledge Catalog can be protected by VPC Service Controls and the product can be used normally inside service perimeters.”
- BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
- BigQuery is an alternative to Dataflow From the official documentation: “To automate the EDA process, use BigQuery or a managed data processing service like Dataflow.”
- BigQuery works with VPC Service Controls From the official documentation: “BigQuery integrates with VPC Service Controls to restrict access to your BigQuery resources from specific networks.”
- Cloud SQL depends on Cloud Storage From the official documentation: “When ingesting data from Cloud SQL, data is first staged to a Cloud Storage bucket.”
- Cloud SQL works with VPC Service Controls From the official documentation: “Make sure that you configure VPC Service Controls so that the project that hosts the CA Service CA pool and CA<br>and the project that hosts Cloud SQL belong to the same service perimeter.”
- Cloud Storage works with clustered tables From the official documentation: “Load data from a CSV file on Cloud Storage to a clustered table.”
- Cloud Storage works with Knowledge Catalog From the official documentation: “To automatically discover and build inventories of the data assets, integrate Knowledge Catalog with your storage systems, such as [BigQuery](/bigquery/docs/introduction), [Cloud Storage](/storage/docs/introduction), and [Pub/Sub](/pubsub/docs/overview).”
- Dataflow works with VPC Service Controls From the official documentation: “To customize DNS resolution when using Dataflow with VPC Service Controls, use Cloud DNS [private zones](/dns/docs/overview#dns-private-zones) instead of using custom BIND servers.”
- Knowledge Catalog depends on Cloud Storage From the official documentation: “Looker (Google Cloud core) publishes LookML metadata to regional Cloud Storage repositories for Knowledge Catalog to consume through a batch sync process.”
- Knowledge Catalog works with VPC Service Controls From the official documentation: “The integration between Looker (Google Cloud core) and Knowledge Catalog supports VPC Service Controls.”
- VPC Service Controls secures Dataflow From the official documentation: “Place your Dataflow pipelines inside a security perimeter using VPC Service Controls.”
- VPC Service Controls secures Datastream From the official documentation: “The API for Datastream can be protected by VPC Service Controls”
How learners score on this topic
Last updated