Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Google Cloud service
BigQuery Storage Read API
Topic in 3 exam domains
- BigQuery Storage Read API · Service
- BigQuery Storage Read API · Feature
An API for fast data retrieval in BigQuery that can run read operations concurrently with mutating transactions.
Where this appears on the exam
3 objectives in 2 of 5 domains
Related topics
This topicWorks alongsideBuilds on theseBuilds on theseBuilds on theseApplied
BigQuery Storage Read APIFeature
Application Default CredentialsFeatureData StudioServicedata maskingFeatureCloud StorageServiceGoogle Cloud StorageServicePub/SubServiceManaged Service for Apache SparkServiceBigQueryServicePrivate Google AccessFeatureDataflowServicePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- BigQuery Storage Read API works with Data Studio From the official documentation: “Data Studio automatically uses the Storage Read API when doing so improves query runtimes.”
- BigQuery Storage Read API works with data masking From the official documentation: “Compatible. Data masking policies are enforced in the BigQuery Storage Read API.”
- BigQuery Storage Read API works with Managed Service for Apache Spark From the official documentation: “In the second way, Managed Service for Apache Spark directly accesses a dataset through the BigQuery API and then processes that ingested data to train a machine learning (ML) model.”
- Dataflow depends on BigQuery Storage Read API From the official documentation: “To read BigLake tables from [Dataflow](/dataflow), use the Dataflow connector in `DIRECT_READ` mode to use the BigQuery Storage API.”
- Application Default Credentials works with Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials secures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- BigQuery depends on Application Default Credentials From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- Dataflow depends on Application Default Credentials From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials works with Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “External BigQuery data source backed by Cloud Storage”
- Application Default Credentials works with BigQuery From the official documentation: “Create a BigQuery client using application default credentials.”
- Application Default Credentials secures Pub/Sub From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- Pub/Sub depends on Application Default Credentials From the official documentation: “To authenticate to Pub/Sub, set up Application Default Credentials.”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- BigQuery works with Data Studio From the official documentation: “Select the BigQuery [connector](/data-studio/connector).”
- Application Default Credentials works with Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Application Default Credentials secures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Application Default Credentials works with Google Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
- Managed Service for Apache Spark depends on Private Google Access From the official documentation: “Managed Service for Apache Spark batch workloads and interactive sessions run on VMs with internal IP addresses only and on a regional subnet with [Private Google Access (PGA)](/vpc/docs/configure-private-google-access) enabled automatically on the subnet.”
- BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
- Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
- BigQuery works with Managed Service for Apache Spark From the official documentation: “This means you can quickly read and write BigQuery data to and from Managed Service for Apache Spark.”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Application Default Credentials secures Dataflow From the official documentation: “To authenticate to Dataflow, set up Application Default Credentials.”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery is an alternative to Managed Service for Apache Spark From the official documentation: “Run queries against your federated data using BigQuery or Managed Service for Apache Spark.”
- Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Managed Service for Apache Spark accesses these files on Cloud Storage.”
- Dataflow is an alternative to Managed Service for Apache Spark From the official documentation: “For more complex data loads into your Cloud SQL for PostgreSQL database, consider using Dataflow or Managed Service for Apache Spark, which involves creating an ETL process.”
- Application Default Credentials works with Managed Service for Apache Spark From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
- Application Default Credentials secures Managed Service for Apache Spark From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
- Google Cloud Storage depends on Application Default Credentials From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- Managed Service for Apache Spark depends on Cloud Storage From the official documentation: “Managed Service for Apache Spark uses Cloud Storage as its default distributed file system.”
- Managed Service for Apache Spark works with Private Google Access From the official documentation: “Managed Service for Apache Spark automatically enables Private Google Access on the regional subnet used by Managed Service for Apache Spark clusters created with image version `2.2` and later.”
- Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
- Dataflow depends on Google Cloud Storage From the official documentation: “- A gcs bucket for staging DataFlow files. Replace `<BUCKET_NAME>`: the name of a valid Google Cloud Storage bucket.”
- Dataflow works with Google Cloud Storage From the official documentation: “Data is transferred between Spanner and Cloud Storage using Dataflow.”
- Managed Service for Apache Spark depends on Application Default Credentials From the official documentation: “To authenticate to Managed Service for Apache Spark, set up Application Default Credentials.”
- Application Default Credentials configures BigQuery From the official documentation: “To authenticate to BigQuery, set up Application Default Credentials.”
- Application Default Credentials configures Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see [Set up authentication for client libraries](/storage/docs/authentication#client-libs).”
- Application Default Credentials secures Google Cloud Storage From the official documentation: “To authenticate to Cloud Storage, set up Application Default Credentials.”
- BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
- BigQuery is an alternative to Dataflow From the official documentation: “To automate the EDA process, use BigQuery or a managed data processing service like Dataflow.”
- BigQuery works with Private Google Access From the official documentation: “Secure API access | Private Google Access | Lets workers reach Cloud Storage and BigQuery privately.”
- Cloud Storage works with Private Google Access From the official documentation: “Secure API access | Private Google Access | Lets workers reach Cloud Storage and BigQuery privately.”
- Data Studio works with Google Cloud Storage From the official documentation: “All the data you upload via Data Studio is stored in Google Cloud Storage.”
- Dataflow depends on Private Google Access From the official documentation: “Workflow failed. Causes: Subnetwork SUBNETWORK on project PROJECT_ID network NETWORK in region REGION does not have Private Google Access, which is required for usage of private IP addresses by the Dataflow workers.”
- Dataflow works with Private Google Access From the official documentation: “Enable Private Google Access and specify one of the following options in your Dataflow job:”
- Google Cloud Storage works with Managed Service for Apache Spark From the official documentation: “Dataproc job outget gets saved to a GCS bucket allocated to it.”
- Google Cloud Storage works with Pub/Sub From the official documentation: “Both Google Managed Service for Apache Kafka and Pub/Sub integrates with various Google Cloud services like Dataflow, BigQuery, Cloud Storage, and more.”
- Google Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- Managed Service for Apache Spark configures Private Google Access From the official documentation: “Managed Service for Apache Spark enables [Private Google Access (PGA)](/vpc/docs/private-google-access) on the specified subnet.”
- Managed Service for Apache Spark works with Pub/Sub From the official documentation: “You can use Managed Service for Apache Spark to process high-volume streams from a service like Pub/Sub or Apache Kafka and land the data in the bronze tier.”
- Private Google Access configures Dataflow From the official documentation: “However, to allow Dataflow workers with private IP addresses to access the external IP addresses that Google Cloud APIs and services use, you must also manually enable Private Google Access for the subnet.”
How learners score on this topic
Last updated