Prepare and test your skills
Prepare and test your skills
Implement Automation Strategies and CI/CD Integration is the practice of running machine learning work without manual steps and delivering code changes safely to production. Vertex AI Pipelines (opens in a new tab) acts as the central orchestrator for the end-to-end workflow from data preparation to training, evaluation, and deployment. Cloud Build (opens in a new tab) acts as the continuous integration and delivery engine for ML code and pipeline definitions. Together they support scheduled retraining, event-driven retraining on new data or drift, and automated testing and deployment of models.
Automated pipeline execution means Vertex AI Pipelines runs start on a schedule or in response to an event instead of a person starting them manually. A scheduled run handles recurring work such as retraining on a fixed interval, while a triggered run responds to new data, code changes, or monitoring signals. Each run uses automated parameterization and versioning so inputs, code, and outputs are recorded in a version history for reproducibility.
Event-driven orchestration connects data and monitoring systems to pipeline triggers so fresh signals start retraining. A data ingestion workflow can trigger a training pipeline after BigQuery (opens in a new tab) ingests new data, so the model learns from the latest dataset without delay. Cloud Logging (opens in a new tab) can publish a message when log filters match new-data arrival or when Vertex AI Model Monitoring emits skew and drift alerts, and that message can start a retraining or fine-tuning pipeline. Pipeline progress and resource usage flow from Vertex AI Pipelines to built-in logging and tracing and to Cloud Monitoring alerts, so failures and drift feed back into continuous training.
Continuous integration and delivery with Cloud Build means ML code changes are built, tested, and packaged into deployable pipeline artifacts automatically. Cloud Build imports source from Artifact Registry, GitHub, or Bitbucket, then executes a build defined in a cloudbuild.yaml configuration file as a series of build steps where each step runs in a Docker container. Builds can start manually or through automated build triggers that fire when source changes are pushed, with separate routines for branches such as development and main and with criteria that limit when a trigger fires.
A typical ML source repository contains all of the inputs Cloud Build needs to reproduce the pipeline:
cloudbuild.yaml scripts, test triggers, and pipeline deployment scriptssettings.yaml with pipeline input parametersCloud Build captures build-time environment values through substitutions such as $COMMIT_SHA, $REPO_NAME, $BRANCH_NAME, $TAG_NAME, $REVISION_ID, $PROJECT_ID, and $BUILD_ID. These substitutions let the same build definition tag images with the exact commit, reuse requests with different values, and trace an image back to the source change that produced it.
The Cloud Build workflow from source commit to pipeline deployment converts tested code into versioned containers and a compiled pipeline definition. The workflow starts when a developer pushes code from a local machine or Vertex AI Workbench to the development branch, which fires the Cloud Build trigger for that branch. The trigger then runs an ordered sequence that only promotes code that passes tests.
A left-to-right then right-to-left process flow showing Cloud Build steps from source copy and tests to building images tagged $COMMIT_SHA, publishing to Artifact Registry, compiling pipeline.json, and running the pipeline to produce a model on Vertex AI Inference.
/workspace directory in the Cloud Build runtime environment.$COMMIT_SHA.component.yaml file to point to the newly built and tagged images.pipeline.json definition file.pipeline.json to Artifact Registry for versioning and reuse.Exam tip: Cloud Build triggers respond to code commits and produce versioned images and pipeline.json, while scheduled Vertex AI Pipelines runs and data or drift events trigger retraining on new data.
Testing and validation in CI/CD proves that infrastructure, dependencies, packaging, model quality, and data are ready for deployment. Cloud Build runs test suites for code and packaging, then the workflow adds stress tests under varying load, bulk evaluations across diverse datasets, and data-integrity checks before retraining. Stress tests can use tools such as Locust, Grafana k6, or Apache JMeter, while latency, error rate, and resource use flow into Cloud Monitoring to reveal bottlenecks.
Validation and continuous training connect Cloud Build to other Google Cloud services in a defined order. First, Vertex AI Pipelines performs automated model evaluation, BigQuery supports large-scale data analysis, and Dataflow (opens in a new tab) validates features and transformations. Then continuous training automatically retrains models on new data or prompts, and Vertex AI Model Garden supplies the latest base models for tuning generative AI systems. For integration and regression testing, the build can submit a pipeline run and call the wait() method to wait for completion before promoting the pipeline, and triggers can be skipped when only documentation or experimentation notebooks change.
Optimize Execution Environments and Infrastructure Governance for Vertex AI Pipelines means running each pipeline step with the right identity, network isolation, and compute resources. Vertex AI Pipelines (opens in a new tab) orchestrates data processing, training, evaluation, and deployment steps. Each step accesses data in Cloud Storage (opens in a new tab), writes artifacts to Model Registry (opens in a new tab), and runs on managed compute. Governance controls who the pipeline acts as, where its traffic can flow, and how much performance and cost each step consumes.
Pipeline identity security in Vertex AI Pipelines is based on dedicated Identity and Access Management (IAM) service accounts that grant only the minimum permissions each stage needs. A training pipeline uses its own service account that can read data from only a specific Cloud Storage bucket and write model artifacts to Model Registry, so a compromise or misconfiguration in one pipeline cannot reach unrelated data or models. The same least-privilege principle applies to Vertex AI custom training jobs, pipeline runs, and deployed models, with central management of users, groups, and roles through IAM.
Access can be further restricted with IAM Conditions that enforce conditional, attribute-based access control on the service account. For example, a condition allows a service account to trigger a Vertex AI pipeline only if the request originates from a trusted Cloud Build trigger, so automation cannot be invoked from an untrusted source. Vertex AI Pipelines also provides low-level access control when MLOps stages integrate with other Google Cloud services and resources. When pipelines are re-executed, explainability evaluations such as feature attributions and responsible AI checks are run before model artifacts are deployed, which helps detect unauthorized changes that indicate tampering, insecure endpoints from misconfiguration, and compromised container images.
Network isolation for Vertex AI Pipelines keeps training data, code, and prediction traffic inside trusted boundaries instead of exposing them to public traffic. Training environments are isolated with dedicated projects or Virtual Private Clouds (VPCs), combined with VPCs, firewalls, and security perimeters to control access to deployed models. Sensitive training workloads can add execution protection with Shielded VMs or Confidential Computing to protect data and code during execution.
For Vertex AI managed training, compute infrastructure connects privately through a Private Service Connect (opens in a new tab) interface that lets custom training jobs communicate privately with other authorized Google Cloud services. Jobs can also use a private IP address for increased network security and lower latency than a public IP address. For online prediction, VPC Service Controls perimeters protect private endpoints and govern access to the underlying models, and private endpoints enable access to models from within a VPC network. IAM is not directly applied to the private endpoint itself, but the target service uses IAM to manage access to the models, and Private Service Connect is recommended for online prediction.
Exam tip: IAM does not attach directly to a Vertex AI private endpoint — the endpoint keeps traffic inside the VPC, while the target Vertex AI service enforces IAM access to the model.
Compute optimization for Vertex AI Pipelines means selecting the machine type, accelerator, and scaling policy that fit each pipeline step rather than using one size for all steps. Custom training jobs define worker pool specifications that control cost and speed, including the machine type, the type and number of accelerators, and the number of worker pools. Training and inference have different accelerator needs, with high-memory training accelerators for large model training and efficient inference accelerators for serving.
The documentation supports the following right-sizing choices:
This separation lets a pipeline spend heavily on a short training step while keeping preprocessing, evaluation, and serving lean. Combined with infrastructure as code through Terraform and reusable templates, the same machine and accelerator choices become repeatable and auditable across runs.
Container strategy for Vertex AI Pipelines isolates tasks with different dependencies into different containers to avoid a single point of failure. For example, feature engineering, training or fine-tuning, and inference each run in a separate container, so library conflicts in one stage do not break another stage. This modular packaging also gives machine learning engineers flexibility to control and customize each execution environment.
Steps can use validated prebuilt containers for standard training and prediction or custom containers when specialized frameworks or dependencies are required. Both container types can serve through private endpoints based on Private Service Connect, with model signatures that define a predefined input and output schema for secure inference. Storing resulting artifacts in Model Registry enables fine-grained access control and encryption at rest, with additional protection from Cloud KMS encryption, Cloud Storage access controls, and digital signatures for packages and containers from approved builds.
Design Modular Pipeline Architectures and Component Reusability in Vertex AI Pipelines means building a machine learning lifecycle as a set of small, reusable steps that run in order. Vertex AI Pipelines (opens in a new tab) is a serverless orchestrator for ML workflows, so teams describe the workflow once as a pipeline and the service runs it without managing servers. This approach standardizes work from data ingestion to model deployment and makes runs scalable, reproducible, and governed.
Modular pipeline building blocks are the three related resources that define, package, and execute a workflow. An ML pipeline is a portable and scalable ML workflow based on containers and Google Cloud services. A pipeline component (opens in a new tab) is a self-contained set of code that performs one step in the workflow, such as data preprocessing, data transformation, or model training. A pipeline job is an execution instance of a pipeline definition, represented by the PipelineJob resource as a set of tasks connected by input-output dependencies.
Data flows from one component task to the next through declared inputs and outputs. The orchestrator builds a directed acyclic graph (DAG) from those dependencies, so a training step runs only after its transformed dataset arrives from the preprocessing step. Because each step runs in its own container, teams can scale, replace, or reuse individual tasks without rewriting the whole workflow.
Reusable components are defined programmatically with the Kubeflow Pipelines (KFP) software development kit (SDK) or TensorFlow Extended (TFX). The Kubeflow Pipelines SDK v2.0 or later uses a domain-specific language to define tasks, pass artifacts between tasks, and compile the pipeline into a YAML file for submission. TensorFlow Extended v0.30.0 or later provides its own pipeline framework with built-in integration for Apache Beam and large-scale TensorFlow processing.
| Option | Version requirement | Choose when |
|---|---|---|
| Kubeflow Pipelines (KFP) | SDK v2.0 or later | For general use cases and custom frameworks such as PyTorch, scikit-learn, or XGBoost |
| TensorFlow Extended (TFX) | v0.30.0 or later | When the workflow uses TensorFlow to process terabytes of structured or text data |
The Google Cloud Pipeline Components (GCPC) SDK accelerates reuse by providing prebuilt, production-quality Kubeflow Pipelines components. Teams load these components from a URL or repository and use them as building blocks instead of writing every step from scratch. Common reusable blocks include downloading data from Cloud Storage, selecting columns with pandas, filling missing values, splitting rows, training a framework-specific model, uploading the model to the registry, and deploying to an endpoint.
Exam tip: Choose TensorFlow Extended when the workload is TensorFlow over terabytes of structured or text data; choose the Kubeflow Pipelines SDK for other frameworks and use cases.
Assembling components from ingestion to deployment means chaining reusable components into one end-to-end lifecycle. The pipeline is defined in code, compiled into YAML, and submitted as a PipelineJob, which the service then orchestrates in dependency order. Dataflow (opens in a new tab)-based preprocessing can be included in three ways, from least to most complex:
A typical modular lifecycle moves artifacts forward through ordered stages:
Vertex ML Metadata is the system for tracking and analyzing metadata from ML workflows. Vertex AI Pipelines automatically tracks ML artifacts with Vertex ML Metadata, so each pipeline run records which dataset, parameters, code, and component version produced which model and metrics. Vertex AI Experiments extends this tracking by recording the steps of a run, the inputs such as algorithms and datasets, and the outputs such as models, checkpoints, and metrics.
Lineage is stored under a MetadataStore, which is the top-level, regionalized container associated with a Google Cloud project. Organizations typically use one shared MetadataStore per project to hold artifacts, executions, and contexts. A MetadataSchema validates the key-value fields on those resources using OpenAPI Schema Objects described in YAML. Because every execution links its inputs to its outputs, teams can reproduce a model version, compare pipeline runs, and audit the full history from ingestion to deployment.
/workspace, runs tests, builds one container per component tagged with $COMMIT_SHA, updates component.yaml, compiles pipeline.json, and uploads artifacts to Artifact Registry.Vertex AI Pipelines is the central orchestrator for the end-to-end workflow from data preparation to training, evaluation, and deployment, while Cloud Build is the continuous integration and delivery engine that builds, tests, and packages machine learning code into versioned containers and pipeline definitions. Together they support scheduled retraining, event-driven retraining on new data or drift, and automated testing and deployment of models.
Use Kubeflow Pipelines SDK version 2.0 or later for general use cases and custom frameworks such as PyTorch, scikit-learn, or XGBoost, and use TensorFlow Extended version 0.30.0 or later when the workflow uses TensorFlow to process terabytes of structured or text data.
Cloud Build copies the source repository to /workspace and runs unit and integration tests, continuing only if the tests pass. It then builds one Docker container image for each pipeline component tagged with the commit SHA, uploads the images to Artifact Registry, updates the image URL in each component.yaml file, compiles the workflow into the pipeline.json definition file, uploads it to Artifact Registry for versioning and reuse, and optionally runs the pipeline for testing or production.
Vertex AI Pipelines uses dedicated Identity and Access Management service accounts with least privilege so each stage can access only what it needs, with Identity and Access Management Conditions able to restrict invocation to trusted automation sources. Network isolation keeps training and prediction traffic inside trusted boundaries using dedicated projects or Virtual Private Clouds, Private Service Connect for private communication between custom training jobs and other Google Cloud services, and Virtual Private Cloud Service Controls perimeters to protect private endpoints for online prediction.
An enterprise is designing an automated machine learning workflow using Vertex AI Pipelines to process sensitive analytical datasets stored in Cloud Storage and BigQuery.
The security and governance team establishes the following operational constraints:
Which architectural approach should you implement to satisfy these security and governance requirements?
Last updated