Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Prepare and test your skills
Advanced
Prepare and test your skills
Concept
transfer configuration
Topic in 1 exam domain
Also called: data transfer configuration, data transfer configurations, transfer, transfer config, transfer configs, transfer configurations, transferConfigs
A specification that defines what data objects to transfer, schedules, credentials, and target datasets for recurring data transfers.
Where this appears on the exam
1 subdomain in 1 of 5 domains
Related topics
FoundationsBuilding blocksBuilding blocksBuilding blocksThis topicWorks alongsideAppliedBigQueryServiceCustomer-Managed Encryption KeysFeaturePub/SubServiceCloud LoggingServiceCloud StorageServiceBigQuery Data Transfer ServiceServiceDataflowServiceCloud SQLService
transfer configurationConcept
Cloud MonitoringServiceservice accountConceptscheduled queryFeaturePoints to what builds on itWorks with or alternative toHover or tap a line to see what connects the two
- BigQuery works with transfer configuration From the official documentation: “Disabling a transfer does **not** remove any data already transferred to BigQuery.”
- transfer configuration is part of BigQuery Data Transfer Service From the official documentation: “Deletes a data transfer configuration in the BigQuery Data Transfer Service.”
- Customer-Managed Encryption Keys configures transfer configuration From the official documentation: “You can update a CMEK for a transfer if the transfer configuration was originally created with a CMEK encryption.”
- Pub/Sub works with transfer configuration From the official documentation: “Create a transfer configuration, where notifications are sent to Cloud Pub/Sub when a transfer run succeeds or fails.”
- transfer configuration configures scheduled query From the official documentation: “To schedule a query, you can use the bq command-line tool to make a transfer configuration.”
- BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
- BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
- Cloud SQL works with Cloud Storage From the official documentation: “Exports data from a Cloud SQL instance to a Cloud Storage bucket as a SQL dump or CSV file.”
- BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
- BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
- Customer-Managed Encryption Keys secures Cloud Storage From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
- Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
- BigQuery works with BigQuery Data Transfer Service From the official documentation: “The BigQuery Data Transfer Service uses load jobs to load Amazon S3 data into BigQuery.”
- BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
- BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
- Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
- Customer-Managed Encryption Keys secures Cloud SQL From the official documentation: “Enable CMEK for Cloud SQL Databases”
- Cloud Logging works with Pub/Sub From the official documentation: “Pub/Sub writes various log entries to Cloud Logging in the Google Cloud console.”
- Cloud Storage works with Customer-Managed Encryption Keys From the official documentation: “Cloud Storage supports CMEK for routing logs.”
- Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
- Cloud Logging works with Cloud SQL From the official documentation: “Database Migration Service and Cloud SQL use Cloud Logging.”
- Cloud Logging works with Cloud Storage From the official documentation: “Cloud Storage writes various log entries to Cloud Logging in the Google Cloud console.”
- Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
- Customer-Managed Encryption Keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
- Customer-Managed Encryption Keys secures Pub/Sub From the official documentation: “Encrypt Pub/Sub topic with CMEK”
- BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
- BigQuery Data Transfer Service works with Cloud Storage From the official documentation: “- [Cloud Storage](/bigquery/docs/cloud-storage-transfer)”
- Cloud Logging works with Cloud Monitoring From the official documentation: “You can store and view query-response activity logs in Cloud Logging and set up logs-based monitoring by using Cloud Monitoring.”
- BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
- BigQuery Data Transfer Service works with Cloud Logging From the official documentation: “The BigQuery Data Transfer Service exports logs and metrics to Cloud Monitoring and Cloud Logging that provide observability into your transfers.”
- BigQuery Data Transfer Service works with Pub/Sub From the official documentation: “If you intend to set up transfer run notifications for Pub/Sub, you must have `pubsub.topics.setIamPolicy` permissions.”
- BigQuery Data Transfer Service works with scheduled query From the official documentation: “- [Scheduled Queries](/bigquery/docs/scheduling-queries#using_a_service_account)”
- BigQuery Data Transfer Service works with service account From the official documentation: “BigQuery Data Transfer Service lets you authenticate data transfers for supported data sources by using a [service account](/iam/docs/service-account-overview) instead of individual user credentials.”
- Cloud Logging is an alternative to Cloud Storage From the official documentation: “Instead, they are routed to either Cloud Storage (recommended) or Cloud Logging, depending on your organization's configuration.”
- Cloud Logging works with Customer-Managed Encryption Keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
- Cloud Monitoring works with Cloud SQL From the official documentation: “You can also get more details by viewing the Cloud Monitoring logs for your Cloud SQL destination instance.”
- Cloud Monitoring works with Dataflow From the official documentation: “Dataflow is fully integrated with Cloud Monitoring.”
- Cloud Monitoring works with Pub/Sub From the official documentation: “Use Cloud Monitoring and Pub/Sub to set up custom monitoring and alerting policies for exported logs.”
- Cloud Storage works with service account From the official documentation: “The service account can manage objects in the Cloud Storage bucket.”
- Customer-Managed Encryption Keys depends on Cloud Storage From the official documentation: “If used for storing Customer-managed Encryption Keys (CMEK): Cloud Storage (See [Cloud Storage pricing](/storage/pricing).)”
- Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
- Pub/Sub works with service account From the official documentation: “5. Creates a service account to represent the Pub/Sub subscription identity.”
- scheduled query is part of BigQuery From the official documentation: “In the [Google Cloud Console](https://console.cloud.google.com/), go to **BigQuery**. 2. In the left navigation, click **Scheduled queries**.”
- scheduled query works with service account From the official documentation: “This service account will be used to execute the scheduled queries.”
- BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
- BigQuery works with service account From the official documentation: “The service account can be used by services, such as Cloud Run, to read and modify data in the BigQuery dataset.”
- BigQuery Data Transfer Service works with Cloud Monitoring From the official documentation: “The BigQuery Data Transfer Service exports logs and metrics to Cloud Monitoring and Cloud Logging that provide observability into your transfers.”
- BigQuery Data Transfer Service works with Cloud SQL From the official documentation: “BigQuery Data Transfer Service lets you transfer data from a private Cloud SQL instance across different Google Cloud projects over an internal IP address.”
- BigQuery Data Transfer Service works with Customer-Managed Encryption Keys From the official documentation: “When you specify a CMEK with a transfer, the BigQuery Data Transfer Service applies the CMEK to any intermediate on-disk cache of ingested data so that the entire data transfer workflow is CMEK compliant.”
- BigQuery Data Transfer Service depends on Pub/Sub From the official documentation: “The communication channel between the agent running on premises and BigQuery Data Transfer Service is by publishing Pub/Sub messages to a per transfer topic.”
- Cloud Logging is an alternative to Cloud Monitoring From the official documentation: “To see the status of your invalidation request or find a definitive list of recently submitted requests, you can use Cloud Logging or Cloud Monitoring.”
- Cloud Logging depends on Customer-Managed Encryption Keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
- Cloud Logging works with Dataflow From the official documentation: “You can control the volume of Dataflow logs ingested by Cloud Logging by following the [Filter logs documentation](/dataflow/docs/guides/filter-logs).”
- Cloud Monitoring works with scheduled query From the official documentation: “Scheduled queries automatically create a metric in Cloud Monitoring.”
- Cloud SQL depends on Cloud Storage From the official documentation: “When ingesting data from Cloud SQL, data is first staged to a Cloud Storage bucket.”
- Cloud SQL works with Dataflow From the official documentation: “Use Datastream to replicate changes from your MySQL instance to either BigQuery or Cloud Storage, and then use Dataflow or Managed Service for Apache Spark to bring the data into your Cloud SQL instance.”
- Cloud Storage works with scheduled query From the official documentation: “To automatically export Gmail logs from BigQuery to Cloud Storage on a recurring schedule, create a scheduled query using the EXPORT DATA statement.”
- Customer-Managed Encryption Keys is part of BigQuery From the official documentation: “If you require more control over encryption keys, BigQuery supports [customer-managed encryption keys](/bigquery/docs/customer-managed-encryption) in the [Cloud Key Management Service](/kms/docs).”
- Customer-Managed Encryption Keys configures Cloud Logging From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
How learners score on this topic
Last updated