Prepare and test your skills
Prepare and test your skills
Model customization and domain adaptation is the practice of tailoring a pre-trained model from Vertex AI Model Garden (opens in a new tab) to an organization's task, vocabulary, and data. A general foundation model that can generate text, images, or code becomes useful for a specialized job only after its behavior is shaped with domain data, prompts, and evaluation. On Gemini Enterprise Agent Platform, that shaping connects base models to data preparation, tuning, pipeline automation, and versioned deployment.
Fine-tuning is the process of continuing the training of a pre-trained foundation model on domain-specific data to improve accuracy for one use case. Most state-of-the-art generative models use the Transformer (opens in a new tab) architecture, in which an encoder converts input text into an intermediate representation and a decoder converts that representation into output, using self-attention to gather context from surrounding words. Training a Transformer from scratch requires significant resources, while fine-tuning a pre-trained Transformer is more efficient. For high-performance training of large language models and diffusion models, the training job depends on accelerated compute, so configure Cloud Tensor Processing Units (TPUs) (opens in a new tab) or GPUs to accelerate both training and inference.
The customization path depends on task complexity, data location, and operational needs:
| Approach | What changes | Choose when |
|---|---|---|
| Fine-tune a Model Garden foundation model | Base model weights are updated with domain data for text, image, or code generation | The task needs specialized accuracy and a pre-trained model matches the domain |
| AutoML on Agent Platform | Architecture selection, hyperparameter tuning, and machine provisioning are handled automatically | Data matches a supported format, and for text, video, or tabular models inference latency greater than 100ms is acceptable |
| Agent Platform custom training | Your own training routine runs and deploys any model type on serverless architecture | The problem does not match BigQuery ML or AutoML criteria, or training must stay consistent with on-premises workflows |
Use Model Garden to select the latest base model available for tuning, so retraining stays current as business needs evolve. After tuning, store the result in Model Registry on Gemini Enterprise Agent Platform, which organizes model versions, tracks lineage, and deploys a chosen version to an endpoint for online or batch inference.
Domain data preparation is the work that makes fine-tuning and reinforcement learning from human feedback reliable, because model quality depends on the data fed into tuning. Data flows from sources such as BigQuery, Dataflow, or Managed Service for Apache Spark into pre-processing and validation, then into a managed dataset or feature store, and finally into the training job. Validation with TensorFlow Data Validation and automated data quality scans in BigQuery checks schemas and detects anomalies before training starts.
For generative AI, data quality has four peer dimensions:
When real-world data is insufficient or imbalanced, generate synthetic data to improve robustness and generalization. Use BigQuery DataFrames to create synthetic datasets from existing patterns or to augment training data, and ensure labels for fine-tuning accurately reflect the quality, relevance, and safety of generated outputs.
Prompt engineering is the practice of designing and managing prompts that guide a generative model without changing its weights, often combined with response filtering and human review. Prompts are developed and managed in Agent Studio on Gemini Enterprise Agent Platform, while Pipelines automates the full adaptation lifecycle so prompt experiments, tuning, and safety checks run the same way every time. This sequence links prompt changes, data changes, and model versions through Vertex ML Metadata and Experiments.
Data flows from preparation through tuning into evaluation and deployment, with the pipeline enforcing order and dependencies:
To keep outputs relevant, enable continuous training so new data or prompts automatically trigger retraining from the latest Model Garden base.
Exam tip: Fine-tuning updates a pre-trained Transformer with domain data and is more efficient than training from scratch, while model distillation trains a smaller student model to mimic a larger teacher model to run efficiently on limited-resource devices.
Model distillation is a technique in which a smaller student model learns from a larger teacher model by training the student to mimic the teacher's output. The student model can then generate new data or make inferences with lower cost and latency, which makes large models more accessible to devices with limited resources. Distillation can also improve generalization by reducing overfitting, so choose distillation when the domain-adapted model is accurate but too large or expensive to serve.
Reliability depends on tracking and monitoring after adaptation. Use Vertex AI TensorBoard to track loss and accuracy over time, visualize the model graph, and compare runs by hyperparameters, and use Feature Store to keep feature data consistent between training and serving. After deployment, Model Monitoring watches for training-serving skew and inference drift and sends alerts when incoming data moves too far from the training baseline. When alerts appear, the monitoring signal flows back to retraining, so the adapted model is refreshed before business performance degrades.
Deployment and API integration strategies define how a model chosen in Vertex AI Model Garden (opens in a new tab) moves into production and how applications call it for predictions. Model Garden is the central library for discovering, testing, customizing, and deploying Google and partner models, while serving is handled by managed infrastructure. A typical strategy registers the model, deploys it to an endpoint for online inference, and connects application requests to that endpoint with security and scaling controls.
The deployment workflow moves a procured Model Garden asset from Google Cloud Marketplace to a serving Vertex AI Endpoint (opens in a new tab) through the registry. The workflow depends on a model artifact in Cloud Storage and a serving container, and the endpoint depends on the registered model version. Deployment is a long-running operation that can take more than five minutes depending on model size.
A four-step process flow showing a Model Garden model moving from Cloud Marketplace procurement through Manage on Vertex AI, upload to Model Registry with a Cloud Storage artifact URI, creation of a Vertex AI Endpoint, and deployment with a machine type, after which applications can send inference requests.
The ordered steps move the model from procurement to serving:
gs://bucket/directory, and a serving container image URI.n1-standard-2.Only after the deploy operation completes can applications send inference requests to the endpoint and receive predictions from the deployed model.
Model Garden groups models into categories that determine how much customization is needed before deployment. The category decides whether an application team can deploy directly, must fine-tune first, or can use a prebuilt solution. Filtering by task, model collection, provider, and features helps match the category to business requirements.
| Category | Description | Deployment approach |
|---|---|---|
| Foundation models | Pretrained multitask large models | Tune or customize for a task, then deploy to an endpoint |
| Fine-tunable models | Models that support custom training | Fine-tune with a custom notebook or pipeline, register the new version, then deploy |
| Task-specific solutions | Prebuilt models for a defined task | Deploy as ready-to-use models, with many customizable on your own data |
Each model card describes the specific model before it is tested, customized, and deployed. Model Garden provides a consistent deployment pattern across categories and built-in integration with tuning, evaluation, and serving.
API integration connects an existing cloud application to the deployed model so request data flows from the application to the Vertex AI Endpoint and prediction data flows back to the application. Vertex AI Inference is the service that uses the trained model to make inferences on new, unseen data after the model is deployed. For agent-based applications, Vertex AI Agent Engine deploys, manages, and scales AI agents so the application calls the agent API while Agent Engine handles production infrastructure.
A database example is Cloud SQL for SQL Server integration (opens in a new tab), where SQL calls flow from the database to an external model endpoint hosted in Vertex AI, Hugging Face, or OpenAI. The database defines an external model object with a model URL and provider, then invokes the AI_GENERATE_EMBEDDINGS function to send text and receive numerical vectors for vector search. Low-latency feature lookup can use Vertex AI Feature Store (opens in a new tab), where feature data flows from BigQuery into an online store and feature views serve values to the model at inference time.
Securing, scaling, and governing inference keeps model APIs private, responsive under load, and limited to approved models. Traffic flows from authorized callers to the endpoint, while access policy and secret storage sit on the trust boundary between the application and the model provider. Scaling is managed by the platform so added endpoint or online-store capacity absorbs higher request volume without changing application code.
Inference governance relies on four peer controls:
Exam tip: Models flagged as unsafe by scans are blocked from deployment, while models flagged as suspicious can still be deployed but require thorough review before use.
Model selection and discovery for specific use cases is the process of finding the right model in Vertex AI Model Garden (opens in a new tab) based on task fit, performance, cost, and trust. Model Garden groups all available Google and partner models in a single location with a consistent deployment pattern for different model types. The platform provides built-in integration with tuning, evaluation, and serving, and it handles deployment and serving for generative AI models that are otherwise difficult to serve.
Model categories in Model Garden describe how ready a model is to use and how it can be adapted for a specific task. Choosing a category is the first architectural decision because it determines whether you prompt, tune, or deploy as-is. Foundation models are large pretrained multitask models, fine-tunable models are designed for custom training workflows, and task-specific solutions are prebuilt for direct use.
| Category | Best for | Customization path |
|---|---|---|
| Foundation models | Multitask large models for broad use cases | Tune or customize for specific tasks using Agent Studio, Agent Platform API, and Agent Platform SDK |
| Fine-tunable models | Models that need adaptation to your data and domain | Fine-tune using a custom notebook or pipeline |
| Task-specific solutions | Prebuilt models that are ready to use | Many can be customized using your own data |
Discovery in Model Garden happens on the Model Garden page in the Google Cloud console, where you browse foundation, tunable, and task-specific models. Each listing has a model card that you open to learn details about the model before testing or deploying it. Filters narrow the catalog to models that match your task requirements, provider trust, and needed features.
Discovery uses four peer filters:
Matching a Google foundation model to task requirements means selecting by the type of output you need to generate. Google provides a set of state-of-the-art foundation models through Vertex AI, including Gemini (opens in a new tab) as the flagship model, alongside specialized models for code, images, and embeddings. This task-first choice avoids using a large general model when a smaller specialized model fits better.
Task matching follows four peer options:
Open-source models can also be deployed to Model Garden when no Google model matches the requirement. The alternative is to self-host on Google Kubernetes Engine (GKE) (opens in a new tab), Cloud Run, or Compute Engine, which is relevant when the model needs a specialized kernel, legacy operating system, or special licensing terms that managed serving does not support.
Evaluating a model for enterprise use requires checking cost, security, and access controls in addition to task fit. For open-source models in Model Garden, you are charged for the compute resources used, not for the model weights themselves. Google performs thorough testing and benchmarking on the serving and tuning containers it provides, with active vulnerability scanning applied to container artifacts.
Cost depends on where compute is consumed:
Third-party models carry additional review steps. Models from featured partners undergo checkpoint scans to ensure authenticity, while third-party models from Hugging Face Hub are scanned by Hugging Face for malware, pickle files, Keras Lambda layers, and secrets. You can also set a Model Garden organization policy at the organization, folder, or project level to allow only vetted models and deny all others.
Exam tip: Models deemed unsafe by scans are blocked from deployment in Model Garden, while models deemed suspicious or able to potentially execute remote code are indicated but can still be deployed after thorough review.
Selection, customization, and deployment form one ordered chain for Model Garden solutions. Selection matches a model category and foundation model to the task, cost, and trust requirements using filters and model cards. Customization then shapes that base model with domain data, prompt iteration, and pipeline automation, with distillation and monitoring controlling cost and reliability. Deployment finally registers the resulting version, places it behind a Vertex AI Endpoint, and connects application traffic to that endpoint under scanning and policy controls.
Fine-tuning updates a pre-trained foundation model's weights with domain-specific data to improve accuracy for one use case, while model distillation trains a smaller student model to mimic a larger teacher model's output for lower cost and latency. Fine-tuning is more efficient than training a Transformer from scratch and is chosen when the task needs specialized accuracy and a pre-trained model matches the domain. Distillation is chosen when the domain-adapted model is accurate but too large or expensive to serve, because the student can generate data or make inferences on devices with limited resources and distillation can improve generalization by reducing overfitting.
The workflow is to select Manage on Vertex AI from the Cloud Marketplace product page to bring the procured model into Vertex AI management, then upload the model to Vertex AI Model Registry with a display name, a Cloud Storage artifact URI and a serving container image URI, then create a Vertex AI Endpoint with a display name, project and location, and finally deploy the registered model to the endpoint with a deployed-model display name and a machine type such as n1-standard-2. Deployment is a long-running operation that can take more than five minutes depending on model size, and only after the deploy operation completes can applications send inference requests to the endpoint and receive predictions.
Match the Google foundation model to the type of output you need to generate, choosing Codey for code generation and Imagen for image generation. Choose text-embedding models when generating embeddings for search, classification, or clustering, and choose Gemini when generating text including multimodal prompts that combine text with images or video and when you want Google's most capable flagship model for text-only prompts.
Models flagged as unsafe by scans are blocked from Model Garden deployment, while models flagged as suspicious can still be deployed but require thorough review before use. Google tests and benchmarks serving and tuning containers with active vulnerability scanning, featured partner models undergo checkpoint scans to verify authenticity, and Hugging Face models are scanned for malware, pickle files, Keras Lambda layers, and secrets. A Model Garden organization policy at the organization, folder, or project level can also allow only vetted models and deny all others.
An enterprise organization is designing an automated assistant in Vertex AI to assist engineering teams with technical documentation and system design reviews. The solution must meet the following requirements:
Which model from Vertex AI Model Garden should the cloud architect choose to satisfy these requirements?
Last updated