Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Concept
BigQuery dataset
Topic in 6 exam domains
- BigQuery dataset · Concept
- BigQuery Dataset · Concept · Security graph node type
Also called: BigQuery dataset where the BigQuery Migration Service writes the results, BigQuery datasets, bigquery_dataset, dataset, Lakehouse dataset
A top-level container in BigQuery used to organize and control access to target replication tables.
Where this appears on the exam
6 subdomains in 3 of 5 domains
Related topics
FoundationsBuilding blocksBuilding blocksThis topicWorks alongsideBigQuery Data ViewerConceptBigQuery Job UserConceptIdentity and Access ManagementServicePub/SubServiceCloud SQLServiceVPC Service ControlsServiceprojectConceptservice accountConceptKnowledge CatalogServiceCloud StorageServiceregionConceptCloud RunServiceBigQueryServiceCloud Storage bucketConcept
BigQuery datasetConcept
Google Cloud MarketplaceServicenamespaceConceptPub/Sub subscriptionConceptCloud MonitoringServicePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- BigQuery dataset is part of BigQuery From the official documentation: “Sample demonstrating how to create a dataset in BigQuery.”
- BigQuery Data Viewer secures BigQuery dataset From the official documentation: “- BigQuery Data Viewer ([`roles/bigquery.dataViewer`](/bigquery/docs/access-control#bigquery.dataViewer)) or any custom role with permissions `bigquery.tables.get` and `bigquery.tables.getData`. When granted on a table or view, this role provides permissions to read data and…”
- BigQuery dataset works with Cloud Run From the official documentation: “The Cloud Run service can interact with the BigQuery dataset.”
- BigQuery dataset works with Cloud Storage bucket From the official documentation: “When you query data in Cloud Storage by using a [BigLake](/bigquery/docs/query-cloud-storage-using-biglake) or a [non-BigLake external table](/bigquery/docs/query-cloud-storage-data), the bucket must be colocated with your BigQuery dataset that contains the external table…”
- BigQuery dataset works with Google Cloud Marketplace From the official documentation: “As a publisher, you can share BigQuery datasets on Cloud Marketplace at scale and configure paid, free, or trial subscription plans and durations without managing individual transactions.”
- BigQuery dataset works with Knowledge Catalog From the official documentation: “If Knowledge Catalog needs to access resources in other projects (for example, Cloud Storage buckets or BigQuery datasets that you want to attach as assets or scan for data profiles), you must grant this service agent the required permissions in the projects containing those…”
- BigQuery dataset works with namespace From the official documentation: “Your Lakehouse namespaces and schemas are also automatically mapped to BigQuery datasets.”
- BigQuery dataset is part of project From the official documentation: “Deleting your project removes all datasets and all tables in the project.”
- BigQuery dataset works with Pub/Sub From the official documentation: “The Pub/Sub subscription can write messages to the BigQuery dataset.”
- BigQuery dataset works with Pub/Sub subscription From the official documentation: “The Pub/Sub subscription can write messages to the BigQuery dataset.”
- BigQuery dataset is part of region From the official documentation: “BigQuery considers datasets, column policies, and slot commitments to be regional resources. Slots can only access datasets in the local region, and column policies can only be applied to tables within local datasets.”
- BigQuery dataset works with service account From the official documentation: “The service account can be used by services, such as Cloud Run, to read and modify data in the BigQuery dataset.”
- BigQuery Job User secures BigQuery dataset From the official documentation: “- BigQuery Job User ([`roles/bigquery.jobUser`](/bigquery/docs/access-control#bigquery.jobUser)) or any custom role with permissions `bigquery.jobs.create`. Provides permissions to run jobs, including queries, within the project using the BigQuery API.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
- BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
- Cloud SQL works with Cloud Storage From the official documentation: “Exports data from a Cloud SQL instance to a Cloud Storage bucket as a SQL dump or CSV file.”
- Cloud Run depends on Pub/Sub From the official documentation: “Cloud Run function triggered by Pub/Sub to fetch HYPR MFA logs and write to GCS.”
- BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
- VPC Service Controls secures BigQuery From the official documentation: “BigQuery data is protected with VPC Service Controls”
- BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
- Cloud Run works with Cloud SQL From the official documentation: “Connecting from Cloud Run (fully managed) to Cloud SQL”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Cloud Run depends on service account From the official documentation: “The Cloud Run function needs a service account with permissions to write to GCS bucket and be invoked by Pub/Sub.”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery Job User secures BigQuery From the official documentation: “- [BigQuery Job User](/iam/docs/roles-permissions/bigquery#bigquery.jobUser) (`roles/bigquery.jobUser`) on the project.”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery Data Viewer secures BigQuery From the official documentation: “- Export data from a BigQuery table: [BigQuery Data Viewer](/iam/docs/roles-permissions/bigquery#bigquery.dataViewer) (`roles/bigquery.dataViewer`)”
- Cloud Run works with Cloud Storage From the official documentation: “Receive events from Cloud Storage in a Cloud Run service”
- Cloud Run works with Identity and Access Management From the official documentation: “For example, a remote backend deployed on Cloud Run can use Identity and Access Management (IAM) to:”
- Cloud Run works with Pub/Sub subscription From the official documentation: “Uses Terraform to create a Pub/Sub subscription to a Cloud Run service”
- Cloud SQL works with Knowledge Catalog From the official documentation: “You can use Knowledge Catalog to organize and enrich the metadata of Cloud SQL resources.”
- Cloud SQL configures region From the official documentation: “When you create a Cloud SQL instance, you choose a [region](/docs/geography-and-regions) where the instance and its data are stored.”
- Cloud Storage bucket is part of Cloud Storage From the official documentation: “Create a Google Cloud Storage bucket”
- Identity and Access Management secures Cloud Storage From the official documentation: “Cloud Storage uses Identity and Access Management (IAM) for authorization.”
- service account is part of Identity and Access Management From the official documentation: “Service accounts function as both principals and resources within IAM.”
- VPC Service Controls secures Cloud Storage From the official documentation: “Storage Transfer Service supports transfers to Cloud Storage buckets protected by VPC Service Controls.”
- BigQuery works with Identity and Access Management From the official documentation: “BigQuery uses Identity and Access Management (IAM) for authorization.”
- Cloud Monitoring works with Cloud Run From the official documentation: “Cloud Monitoring records metrics regarding the execution of Cloud Run.”
- Cloud Monitoring works with Cloud SQL From the official documentation: “You can also get more details by viewing the Cloud Monitoring logs for your Cloud SQL destination instance.”
- Cloud Monitoring works with Pub/Sub From the official documentation: “Use Cloud Monitoring and Pub/Sub to set up custom monitoring and alerting policies for exported logs.”
- Cloud Run works with Pub/Sub From the official documentation: “A Cloud Run app receives the Pub/Sub message.”
- Cloud Storage works with Knowledge Catalog From the official documentation: “Knowledge Catalog is an intelligent governance solution for data and AI assets in Google Cloud. It provides a centralized inventory where you can discover, manage, and govern your data across Google Cloud data sources like BigQuery, Cloud Storage, Pub/Sub, and Spanner.”
- Cloud Storage works with service account From the official documentation: “The service account can manage objects in the Cloud Storage bucket.”
- Identity and Access Management works with VPC Service Controls From the official documentation: “VPC Service Controls perimeters and Identity and Access Management (IAM) roles work together.”
- Knowledge Catalog works with Pub/Sub From the official documentation: “Knowledge Catalog is an intelligent governance solution for data and AI assets in Google Cloud. It provides a centralized inventory where you can discover, manage, and govern your data across Google Cloud data sources like BigQuery, Cloud Storage, Pub/Sub, and Spanner.”
- Pub/Sub works with service account From the official documentation: “5. Creates a service account to represent the Pub/Sub subscription identity.”
- BigQuery works with Cloud Run From the official documentation: “You can then use Cloud Run to transcode data present in the bucket and move it to BigQuery.”
- BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
- BigQuery works with Cloud Storage bucket From the official documentation: “BigQuery lets you load [Datastore](/datastore) entities from a Cloud Storage bucket into a BigQuery table for analysis.”
- BigQuery works with service account From the official documentation: “The service account can be used by services, such as Cloud Run, to read and modify data in the BigQuery dataset.”
- BigQuery works with VPC Service Controls From the official documentation: “BigQuery integrates with VPC Service Controls to restrict access to your BigQuery resources from specific networks.”
- BigQuery Data Viewer configures BigQuery From the official documentation: “- [BigQuery Data Viewer](/bigquery/docs/access-control#bigquery.dataViewer) (`roles/bigquery.dataViewer`) on projects to which Dataform needs read-only access.”
- BigQuery Data Viewer configures service account From the official documentation: “Vertex AI Feature Store assigns the [BigQuery Data Viewer](/bigquery/docs/access-control#bigquery.dataViewer) Identity and Access Management (IAM) role to this service account.”
- BigQuery Job User secures Knowledge Catalog From the official documentation: “Grant the **BigQuery Job User** (`roles/bigquery.jobUser`) role so Knowledge Catalog can run query jobs in your project:”
- Cloud Monitoring works with Google Cloud Marketplace From the official documentation: “Some virtual machine (VM) products from Cloud Marketplace might include Cloud Monitoring, which collects metrics, events, and metadata from your deployment.”
- Cloud Monitoring works with Knowledge Catalog From the official documentation: “Cloud Logging and Cloud Monitoring let you track billable token consumption for Knowledge Catalog (formerly Dataplex Universal Catalog) data insights scans.”
- Cloud Monitoring works with VPC Service Controls From the official documentation: “Cloud Monitoring performs the VPC perimeter check when a project is *added* to a metrics scope:”
- Cloud Run works with service account From the official documentation: “Map a custom Service Account to a Cloud Run service”
How learners score on this topic
Last updated