Task

Related topics

• BigQuery works with Task From the official documentation: “To load CSV data from Cloud Storage into a BigQuery table that uses column-based time partitioning:”
• Cloud Storage works with Task From the official documentation: “To load CSV data from Cloud Storage into a BigQuery table that uses column-based time partitioning:”
• Job configures Task From the official documentation: “A Job describes the Task to execute and automatically manages Task creation.”
• Task is part of DAG From the official documentation: “A *DAG* is a collection of tasks that you want to schedule and run, organized in a way that reflects their relationships and dependencies.”
• Task is part of Job From the official documentation: “A Batch job consists of one or several tasks.”
• BigQuery is an alternative to Cloud Storage From the official documentation: “The `import_result_sink` must be a Cloud Storage path or a BigQuery table.”
• BigQuery works with Cloud Storage From the official documentation: “Sample to load Avro data from Cloud Storage into a new BigQuery table”
• BigQuery works with Cloud SQL From the official documentation: “Add credentials to connect BigQuery to Cloud SQL.”
• Cloud SQL works with Cloud Storage From the official documentation: “Exports data from a Cloud SQL instance to a Cloud Storage bucket as a SQL dump or CSV file.”
• Cloud Run depends on Pub/Sub From the official documentation: “Cloud Run function triggered by Pub/Sub to fetch HYPR MFA logs and write to GCS.”
• Dataflow works with Pub/Sub From the official documentation: “Pub/Sub notifies Dataflow about the upload.”
• BigQuery works with Dataflow From the official documentation: “Read from BigQuery to Dataflow”
• Cloud Storage works with Dataflow From the official documentation: “Write from Dataflow to Cloud Storage”
• Dataflow works with Managed Service for Apache Airflow From the official documentation: “Launch Dataflow pipelines with Cloud Composer”
• BigQuery works with Knowledge Catalog From the official documentation: “Some Knowledge Catalog functionalities trigger job execution using BigQuery.”
• Cloud Run works with Cloud SQL From the official documentation: “Connecting from Cloud Run (fully managed) to Cloud SQL”
• BigQuery works with Pub/Sub From the official documentation: “createBigQuerySubscription creates a Pub/Sub subscription that exports messages to BigQuery.”
• Cloud Storage is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Storage works with Pub/Sub From the official documentation: “| **Google Cloud** | Cloud Storage | Pub/Sub |”
• BigQuery works with Managed Service for Apache Airflow From the official documentation: “You can use Managed Airflow to orchestrate services in your data pipelines, such as triggering a job in BigQuery or starting a Dataflow pipeline.”
• BigQuery is an alternative to Pub/Sub From the official documentation: “To keep audit logs for a longer period of time or to use more powerful search capabilities, you can route copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Run works with Cloud Storage From the official documentation: “Receive events from Cloud Storage in a Cloud Run service”
• Cloud SQL works with Knowledge Catalog From the official documentation: “You can use Knowledge Catalog to organize and enrich the metadata of Cloud SQL resources.”
• Cloud Run works with Pub/Sub From the official documentation: “A Cloud Run app receives the Pub/Sub message.”
• Cloud Storage works with Knowledge Catalog From the official documentation: “Knowledge Catalog is an intelligent governance solution for data and AI assets in Google Cloud. It provides a centralized inventory where you can discover, manage, and govern your data across Google Cloud data sources like BigQuery, Cloud Storage, Pub/Sub, and Spanner.”
• Cloud Storage works with Managed Service for Apache Airflow From the official documentation: “When you upload your DAG file to the `dags/` folder in Cloud Storage, Managed Airflow parses the file.”
• Dataflow depends on Cloud Storage From the official documentation: “Begin by creating a Cloud Storage bucket using the Google Cloud console or Google Cloud CLI. The Dataflow pipeline uses this bucket as a temporary storage location.”
• Knowledge Catalog works with Pub/Sub From the official documentation: “Knowledge Catalog is an intelligent governance solution for data and AI assets in Google Cloud. It provides a centralized inventory where you can discover, manage, and govern your data across Google Cloud data sources like BigQuery, Cloud Storage, Pub/Sub, and Spanner.”
• Managed Service for Apache Airflow depends on Cloud SQL From the official documentation: “Managed Airflow uses Cloud SQL to run Airflow database.”
• BigQuery works with Cloud Run From the official documentation: “You can then use Cloud Run to transcode data present in the bucket and move it to BigQuery.”
• BigQuery depends on Cloud Storage From the official documentation: “BigQuery relies on Cloud Storage for staging file data and supports reading from and writing to several other sources and Google Cloud services.”
• Cloud SQL depends on Cloud Storage From the official documentation: “When ingesting data from Cloud SQL, data is first staged to a Cloud Storage bucket.”
• Cloud SQL works with Dataflow From the official documentation: “Use Datastream to replicate changes from your MySQL instance to either BigQuery or Cloud Storage, and then use Dataflow or Managed Service for Apache Spark to bring the data into your Cloud SQL instance.”
• Cloud SQL works with Managed Service for Apache Airflow From the official documentation: “For Private IP environments, Managed Airflow assigns only private IP ([RFC 1918](https://www.ietf.org/rfc/rfc1918.txt)) addresses to the managed Google Kubernetes Engine and Cloud SQL VMs in your environment.”
• DAG works with Dataflow From the official documentation: “Now you will reference the files you created earlier to create a DAG that kicks off the Dataflow workflow.”
• DAG depends on Managed Service for Apache Airflow From the official documentation: “This guide shows you how to write an Apache Airflow directed acyclic graph (DAG) that runs in a Managed Service for Apache Airflow environment.”
• DAG works with Pub/Sub From the official documentation: “You can then create a DAG that pulls Pub/Sub messages and then handles these messages.”
• Job is part of BigQuery From the official documentation: “Every time you [load](/bigquery/docs/loading-data), [export](/bigquery/exporting-data-from-bigquery), [query](/bigquery/docs/running-queries), or [copy data](/bigquery/docs/managing-tables#copy-table), BigQuery automatically creates, schedules, and runs a job that tracks the…”
• Job is part of Cloud Run From the official documentation: “On Cloud Run, your code can run as a *service*, *job*, *worker pool*, or *instance*.”
• Job is part of Dataflow From the official documentation: “In Dataflow, a [`Job`](/dataflow/docs/reference/rest/v1b3/projects.jobs#Job) resource represents a Dataflow job.”
• Job works with Knowledge Catalog From the official documentation: “When you run a data quality scan, Knowledge Catalog creates a job.”
• Managed Service for Apache Airflow depends on BigQuery From the official documentation: “This happens because Google Kubernetes Engine depends on BigQuery and Managed Service for Apache Airflow inherits this dependency from the Kubernetes Engine API.”