Custom Training, Kubeflow and AutoML Workflows

Vertex AI custom training is a serverless training mode where user-supplied code runs on managed virtual machines (VMs) defined by a CustomJob resource. The CustomJob carries a job_spec containing one or more worker_pool_specs, and each worker pool defines a machine_spec, a replica_count, and either a container_spec or a python_package_spec. The Vertex AI SDK submits this specification to a regional API endpoint, and Vertex AI provisions the compute, pulls the code artifact, and runs training to produce a model artifact registered on the Models page.

1.1Custom Container Packaging and Prerequisites

Custom container packaging is the method that bundles customized training code into a Docker image that Vertex AI runs directly. The workflow depends on Artifact Registry for hosting the image, Cloud Storage for staging data and outputs, and Identity and Access Management (IAM) for the service account identity. Training data flows from Cloud Storage buckets into the training container, and trained outputs flow from the container back to Cloud Storage and Model Registry.

The prerequisite workflow runs in a fixed order:

1. Create Docker repository and config: Create an Artifact Registry Docker repository to store the custom training image.
2. Build custom container image: Build a container image that contains the customized training code and push the image URI to the repository.
3. Set up service account and Cloud Storage buckets: Configure a service account for job permissions and create Cloud Storage buckets for inputs, TensorBoard logs, and model outputs.
4. Create and launch custom training job: Submit a CustomJob that references the image URI in container_spec and monitor progress in near real time in Vertex AI TensorBoard (opens in a new tab).

Because the container owns its operating system, libraries, and entrypoint, this path supports any framework, system dependency, or custom binary that does not fit a Google-provided image.

1.2Submitting CustomJobs With the Vertex AI SDK

Vertex AI SDK submission is the process that creates a regional JobServiceClient and calls create_custom_job with a parent resource and a job specification. The client is initialized once with client_options set to a regional endpoint such as us-central1-aiplatform.googleapis.com, then reused for multiple requests. The parent names both sides of the placement boundary as projects/{project}/locations/{location}, which determines where the control-plane request routes and where training VMs run.

A custom container job defines display_name and job_spec.worker_pool_specs with explicit compute and container fields. The machine_spec sets machine_type such as n1-standard-4, plus accelerator_type such as NVIDIA_TESLA_K80 and accelerator_count such as 1 when accelerators are needed. The container_spec sets image_uri to the pushed custom image, with command and args controlling the container entrypoint; empty arrays use the image default. The replica_count sets how many VMs in that pool run the container, so replica_count: 1 is single-node training and higher counts scale the pool for distributed training.

1.3Pre-Built Versus Custom Container Environments

The training environment choice determines who builds the container image and how code is packaged. A pre-built environment uses a Google-provided executor image and a Python source package, while a custom container environment uses a user-built image that already contains the code.

Choose the pre-built Python package when standard TensorFlow or PyTorch CPU images are sufficient and the training app can be shipped as a Python module tarball from Cloud Storage. Choose a custom container when the job needs custom system packages, a specific framework build, non-Python code, or full control over startup behavior. The same worker_pool_specs structure carries either choice, so compute, accelerator, and replica decisions remain consistent across both paths.

1.4Distributed Training and Accelerator Configuration

Distributed training configuration lives in worker_pool_specs, where each pool binds compute, accelerators, and code together. The machine_spec on each pool selects the VM family and accelerators, the replica_count on each pool selects how many identical VMs run, and the container_spec or python_package_spec on each pool selects what each replica executes. Scaling from one replica to many replicas increases parallelism but requires training code written to coordinate across workers and to use accelerators when configured.

Accelerator selection pairs machine_type with accelerator_type and accelerator_count on each VM in the pool. Training code must already be written to use graphics processing units (GPUs) before the pool requests them, otherwise the GPUs remain idle. Vertex AI supports the following GPU types for serverless training:

• NVIDIA_GB200, includes GPUDirect-Remote Direct Memory Access (RDMA)
• NVIDIA_B200, includes GPUDirect-RDMA
• NVIDIA_H100_MEGA_80GB, includes GPUDirect-TCPXO
• NVIDIA_H100_80GB
• NVIDIA_H200_141GB, includes GPUDirect-RDMA
• NVIDIA_A100_80GB
• NVIDIA_TESLA_A100, NVIDIA A100 40GB
• NVIDIA_TESLA_P4
• NVIDIA_TESLA_P100
• NVIDIA_TESLA_T4
• NVIDIA_TESLA_V100
• NVIDIA_L4
• NVIDIA_RTX_PRO_6000

Using smaller machine types such as n1-highmem-2 with GPUs can cause logging to fail for some workloads because of central processing unit (CPU) constraints. If a GPU training job stops returning logs, selecting a larger machine type restores log delivery without changing the accelerator request itself.

1.5Monitoring, Debugging, and Model Output

Monitoring connects training logs, interactive debugging, and experiment visualization to the running CustomJob. Training logs flow from each training container to Cloud Logging and are viewed by opening the Training page in the Google Cloud console, selecting the job name, and clicking View logs. An interactive shell can attach to running training containers while the pipeline runs to inspect files, processes, and environment state without stopping the job.

Vertex AI TensorBoard provides near-real-time curves for loss, metrics, model graphs, and profiling data once the job streams summaries to the configured Cloud Storage location. After submission, job progress is tracked on the Vertex AI Experiments page, and profiling with Cloud Profiler helps pinpoint bottlenecks to train faster and at lower cost. When serverless training completes, the trained model appears on the Models page, where it can be deployed to an endpoint, tuned further with hyperparameter tuning, or scheduled based on resource availability.

Kubeflow on Google Kubernetes Engine (GKE) (opens in a new tab) is the approach for running containerized model training as orchestrated, multi-step workflows on Kubernetes instead of single ad-hoc jobs. GKE provides the cluster control plane and accelerator node pools, while pipeline steps run as containers that read data and write predictions, checkpoints, and models to Cloud Storage. This organization makes training scalable, reproducible, and resource-efficient because infrastructure, workflow steps, and artifacts are all versioned and automated together.

2.1Containerized Training Workloads on GKE

Containerized training on GKE packages training code, libraries, and dependencies into container images that run as Pods on GPU or TPU nodes. A TensorFlow training container deployed to a GKE Standard cluster, for example, pulls input data, uses allocated accelerators for computation, and then writes predictions and model outputs to a Cloud Storage bucket for downstream use. GKE Standard mode requires explicit creation and management of clusters and GPU node pools, while GKE Autopilot mode reduces setup steps by managing node provisioning automatically.

The training workload depends on the node pool for accelerators, depends on Cloud Storage for durable data and artifact storage, and depends on Identity and Access Management (IAM) roles for permission to select projects, enable APIs, and access buckets. Workload organization separates stable platform configuration from per-experiment code. Platform teams manage the cluster, node pools, quotas, and storage buckets, while data scientists submit containerized pipeline steps that request specific CPU, memory, and accelerator resources. Because each step runs in its own container with declared resource requests, the Kubernetes scheduler places training Pods only on nodes that satisfy those requests, and failed steps can be retried without rebuilding the cluster.

2.2GPU and TPU Node Pool Provisioning and Capacity Strategy

GPU and TPU node pool provisioning on GKE is the process of defining which accelerators are available for training and how capacity is obtained under cost and availability constraints. Dynamic Workload Scheduler (DWS) (opens in a new tab) powers on-demand accelerator obtainability through Flex-start and Calendar modes, while ComputeClasses define a hierarchy of node configurations that GKE prioritizes during autoscaling. GKE offers NVIDIA H100, A100, and L4 GPUs, plus A3 Ultra H200 and A4 B200 configurations for large-scale training, with L4 GPUs commonly used for inference and A100, H100, H200, or B200 GPUs used for training. For complex multi-GPU topology and colocation, Cluster Director manages accelerator groups across GKE and Slurm, and Ray on Vertex AI abstracts distributed compute so applications request resources without directly managing VMs and containers.

Capacity choices trade stability against cost and preemption risk:

Use Spot VMs combined with Flex-start as a fallback when Spot capacity is unavailable, and define ComputeClasses profiles that prefer reserved GPUs or TPUs first and then fall back to Spot VMs. Plan quotas proactively by analyzing historical utilization in Cloud Monitoring, running load tests to find bottlenecks, and requesting quota increases with forecasting data.

2.3Orchestrating Multi-Step Workflows With Infrastructure as Code

Orchestration with infrastructure as code (IaC) defines GKE clusters, storage, and serving resources in versioned templates so multi-step training workflows are repeatable and promotable across environments. Terraform is the declarative tool for provisioning AI infrastructure, and Cluster Toolkit provides Google-curated Terraform blueprints for ready-to-use high-performance computing (HPC), AI, and machine learning (ML) clusters on Slurm or GKE. Config Connector then manages Google Cloud resources as Kubernetes objects from YAML manifests applied to the GKE cluster, so datasets, models, endpoints, Cloud SQL instances, Pub/Sub topics, and Cloud Storage buckets evolve with the same workflow as application code.

Core IaC resources for a training platform include:

• GKE clusters configured with workload-optimized node pools, such as A100, H100, H200, or B200 GPUs for training and L4 GPUs for inference
• Vertex AI endpoints configured for model serving with defined machine types and scaling policies
• Cloud Storage buckets for datasets, model weights, checkpoints, and prediction outputs

Manage ongoing changes declaratively by storing modular Terraform configurations in version control, templating environment differences, and integrating provisioning into continuous integration and continuous delivery (CI/CD) pipelines with Cloud Build and Infrastructure Manager. A pipeline first runs validate and plan for automated testing, then applies changes on code commit, stores configurations as versioned artifacts, and promotes the same code through distinct dev, staging, and prod configurations. IAM policies and service accounts are also implemented in IaC so each pipeline step receives only the permissions needed to read data and write artifacts.

2.4Experiment Tracking, Artifact Lineage, and Continuous Validation

Experiment tracking and artifact lineage on Google Cloud is the system for recording what was trained, with what data and code, and where the resulting model is stored. Vertex AI Experiments automatically logs parameters, code versions, dataset versions, and results for each training and evaluation run, while Model Registry registers versioned trained artifacts after each successful run or stores them directly in Cloud Storage. This lineage connects a trigger, such as new data arrival, data drift, or a schedule, to the exact training container, hyperparameter tuning trial, distributed training configuration, and evaluation score that produced a model.

Continuous training pipelines enforce validation before promotion through an ordered lifecycle:

1. Trigger: New data, monitoring signal, or schedule starts the pipeline.
2. Training: Vertex AI custom training runs distributed training or hyperparameter tuning trials, including fine-tuning jobs for foundation models.
3. Versioning: Successful outputs are versioned and stored in Cloud Storage or Model Registry.
4. Evaluation: Vertex AI model evaluation checks accuracy, error rate, F1 score, generated-output quality, factual accuracy, safety attributes, and style compliance against minimum thresholds.
5. Promotion decision: Only a model meeting all thresholds is considered for deployment to an endpoint.

Monitor both model behavior and platform health with Cloud Logging for centralized logs, Cloud Monitoring and Google Cloud Managed Service for Prometheus for metrics like queries per second (QPS), token throughput, and first-token latency, Error Reporting for exceptions, Cloud Trace for distributed latency, and Cloud Profiler for CPU and memory bottlenecks. Track deployment status including canary releases and rollbacks with Cloud Deploy integrated with Cloud Monitoring, and set custom alerts for business thresholds integrated with incident tools such as PagerDuty or Slack.

Automated machine learning and tabular workflows on Vertex AI (opens in a new tab) are managed approaches for training models on structured data without manually designing architectures or provisioning infrastructure. Vertex AI AutoML provides prebuilt training routines for common problems such as tabular regression, classification, and forecasting, handling architecture search, preprocessing, hyperparameter tuning, and machine provisioning. Data flows from BigQuery tables or managed datasets to the managed training service, and the resulting model flows to Vertex AI endpoints for serving or to an external device for edge deployment.

3.1What Vertex AI AutoML Automates

Vertex AI AutoML is a managed training environment that automates nearly all aspects of training and serving for supported data types and model objectives. When a job starts, the service selects candidate architectures, applies built-in transformations for preprocessing, runs hyperparameter search, provisions training infrastructure, and packages the best model for deployment. This automation removes manual iteration over architectures, input configurations, and hardware choices, which is evaluated through aggregate metrics such as accuracy, precision, and recall on held-out test data.

AutoML is appropriate when data and problem match a supported format with supported data types and objectives, and when the model can be served from Google Cloud or deployed to an external device. For text, video, or tabular models, the application must tolerate inference latencies greater than 100ms. AutoML tabular models can also be trained from the BigQuery ML environment, which keeps tabular data and training in one system.

3.2SDK Classes for Structured Data Training

Vertex AI SDK for Python (opens in a new tab) training classes are the job objects that create, train, and return models, plus supporting classes for tuning and orchestration. A tabular workflow depends on a dataset resource such as TabularDataset, a training job class, and a run() call that links the dataset, target column, and splits to the returned model. Use HyperparameterTuningJob to tune a training job's hyperparameters with Vertex AI Vizier, and use PipelineJob to automate and monitor the end-to-end machine learning workflow.

Structured-data AutoML in the SDK includes the following peer job classes:

• AutoMLTabularTrainingJob: creates, trains, and returns an AutoML tabular model for classification or regression on managed tabular data.
• AutoMLForecastingTrainingJob: trains a forecasting model with the AutoML training method, which is the best choice for most forecasting use cases.
• SequenceToSequencePlusForecastingTrainingJob: trains forecasting with the Seq2seq-plus training method when sequence-to-sequence behavior improves predictions.
• TemporalFusionTransformerForecastingTrainingJob: trains forecasting with the temporal fusion transformer method when that architecture fits the use case.
• TimeSeriesDenseEncoderForecastingTrainingJob: trains forecasting with a dense-encoder method for time-series data.

AutoMLTabularTrainingJob is configured with a display name, an optimization_prediction_type such as regression, and an optimization_objective such as minimize-rmse. The run() call then takes the dataset, a target_column name, and split fractions such as training_fraction_split and validation_fraction_split to control how managed data flows into training, validation, and test partitions.

3.3Automated Feature Engineering and Data Preparation

Automated feature engineering in this workflow means preprocessing is handled inside the platform rather than in hand-written training code. If the model is built with BigQuery ML, the transformations built into BigQuery ML preprocess the data, and if the model is built with AutoML, the transformations built into AutoML preprocess the data. For custom models, using BigQuery transformations may still be the most cost-effective method before data flows to the training service.

Structured and semi-structured data should be stored in BigQuery, with intermediate processed data also stored in BigQuery for reuse. For maximum training speed, store materialized tables instead of using views or subqueries as training inputs, and consider partitioning for large datasets. After preprocessing, a managed dataset in Vertex AI creates a clear link between data and custom-trained models through Vertex ML Metadata, provides descriptive statistics, and supports automatic or manual splitting into train, test, and validation sets.

3.4Training Budgets, Objectives, and Experiment Tracking

Training budgets and quality in AutoML tabular workflows are controlled through the optimization type, optimization objective, and data splits defined in the SDK. The optimization_prediction_type defines the task, such as regression or classification, while the optimization_objective defines the metric the search optimizes, such as minimize-rmse. The training_fraction_split, validation fraction, and test fraction define what portion of rows from the TabularDataset flows to each phase, so evaluation on test data measures generalization rather than memorization.

Hyperparameter tuning is the automated enhancer provided by the managed training service, which uses Google Cloud processing infrastructure and Vertex AI Vizier to test different hyperparameter configurations across training runs. Tuning removes manual adjustment over numerous runs to reach optimal values and is tracked with Vertex AI TensorBoard and Vertex AI Experiments. TensorBoard visualizes metrics such as loss and accuracy over time, model graphs, and embeddings, while Experiments logs linkage across parameters, metrics, and dataset and model artifacts in Vertex ML Metadata.

3.5Choosing Between Prebuilt and Custom Training

Prebuilt routines and custom training divide by data location, SQL comfort, problem fit, and control needs. BigQuery ML brings data, infrastructure, and predefined model types into a single SQL system, AutoML handles architecture, tuning, and provisioning for supported formats, and custom training on serverless architecture supports any model type with additional services for tuning and monitoring.

Choose custom training when lineage control, custom splitting in training code, or framework consistency across platforms outweighs automation. Choose managed datasets when lineage between data and model and reproducible splits matter, and skip them when custom splitting logic or minimal tracking is preferred.

Key Facts

• Vertex AI custom training runs user code as a CustomJob where each worker_pool_specs entry binds machine_spec, replica_count, and container_spec or python_package_spec together.
• GPU training pools must use an A2, N1, or G2 machine type, and training code must already be written to use GPUs or the accelerators remain idle.
• GKE training separates platform management of clusters, node pools, and Cloud Storage buckets from per-experiment containers that request CPU, memory, and accelerators.
• Dynamic Workload Scheduler Flex-Start supports batch training up to 7 days without a fixed start date, while Calendar mode reserves H100, H200, and Trillium capacity up to 3 months.
• AutoMLTabularTrainingJob uses optimization_prediction_type, optimization_objective such as minimize-rmse, target_column, and split fractions to control tabular training and evaluation.