Prepare and test your skills
Prepare and test your skills
Infrastructure is the bottom layer of the generative AI landscape that supplies compute, storage, networking, and orchestration for training and serving models. Teams do not manage physical hardware directly because managed cloud services provision accelerators, containers, and endpoints on demand. The main choices are Vertex AI (opens in a new tab) for end-to-end machine learning (ML), AI Hypercomputer (opens in a new tab) for large-scale accelerated training, and serverless or Kubernetes platforms for serving.
A layered stack diagram of the generative AI landscape showing five layers: Applications with grounded support and horizontal or vertical apps; Agents with planning, memory, and tool calling; Platforms with Vertex AI pipelines, agents, and grounding; Models with Model Garden offerings like Gemini, Imagen, Claude, and Gemma; and Infrastructure with GKE, Cloud Run, Compute Engine TPUs and GPUs, AI Hypercomputer, and BigQuery, each layer building on the one below.
Managed resources on Vertex AI are the notebooks, datasets, pipelines, containers, and endpoints that Vertex AI creates and operates for the team. Development starts in Colab Enterprise, a collaborative managed notebook environment with BigQuery integration, or in Vertex AI Workbench, a unified notebook environment with scalable infrastructure and Cloud Storage integration. Teams store inputs in a managed dataset hosted by Vertex AI and run repeatable work in Vertex AI Pipelines, which are portable container-based ML workflows. Deployment uses prebuilt containers with common frameworks or custom container images that package code, libraries, and configuration for reproducibility, plus persistent resources such as Ray clusters that stay allocated until deleted to reduce startup time between jobs.
Trained models are deployed to endpoints that serve inferences, which are outputs calculated by running new data through a model. Cloud Run GPUs serve pre-trained or custom self-deployed models for AI inference from containers or source code. Inference runs in two forms: batch inference processes a group of requests and writes results to one file, while online inference returns synchronous results for individual data points that need real-time answers. Supporting services make this reliable: BigQuery analyzes massive datasets with SQL, Cloud Logging collects and monitors logs, Private Service Connect carries traffic from a customer network to Google services without crossing the public internet, and Identity and Access Management (IAM) with service accounts controls who and what can access resources.
Specialized compute for AI workloads is hardware designed to train and serve large models efficiently, mainly graphics processing units (GPUs) and Tensor Processing Units (TPUs). Compute Engine creates and manages Cloud TPU virtual machines (VMs) as single instances or as members of a managed instance group (MIG) that handles creation, scaling, and management. Google Cloud TPUs use hardware built for large matrix operations with on-chip high-bandwidth memory for larger models, and TPUs can be joined in groups called slices that scale with little code change. Teams can also run TPU capacity on Google Kubernetes Engine (GKE) or connect multiple TPU slices over the Data Center Network for very large training jobs.
Single-host and multi-host TPU shapes differ in whether VMs work alone or together. Single-host TPU VMs in a MIG each handle their own separate workload without depending on other VMs, so adding VMs adds more independent capacity for inference. A multi-host TPU slice is a group of interconnected TPU VMs that work together as one unit on a single workload, so data and computation flow across the connected VMs and they depend on each other to finish. AI Hypercomputer manages this at scale by combining accelerators, open software such as PyTorch and JAX, and consumption options such as Spot VMs and Dynamic Workload Scheduler to maximize goodput, the amount of productive ML work completed.
Exam tip: Use single-host TPU VMs in a MIG for multiple independent inference workloads, and use a multi-host TPU slice in a MIG when multiple interconnected TPU VMs must work together on one workload.
Infrastructure selection is the decision about where models are trained, served, and connected to data, and it directly affects total cost of ownership, latency, and time-to-market. The three main Google Cloud options trade control for management effort and speed:
| Platform | Management level | Best-fit workload |
|---|---|---|
| Google Kubernetes Engine (GKE) | Google-managed Kubernetes with deep control and portability | Large-scale training and inference and custom high-performance platforms |
| Vertex AI | Fully managed end-to-end MLOps platform | Rapid development and fast time-to-value without managing infrastructure |
| Cloud Run | Fully managed serverless platform for containers | Event-driven apps and smaller models with scale-to-zero serving |
GKE controls cost at scale with Spot VMs and Flex-start VMs, reduces GPU startup time with faster nodes, and serves low-latency inference with model-aware routing through Inference Gateway, but it requires Kubernetes expertise. Cloud Run scales instances automatically on demand and can scale to zero, so teams pay only while serving and can move quickly from a prototype to production with minimal configuration. Vertex AI offloads setup and operations to accelerate development, which shortens time-to-market for teams focused on models rather than clusters.
Continue with this study guide: Infrastructure
Models are the foundation-model layer that turns data into generated text, code, images, and multimodal outputs. A foundation model is a large model trained on vast data that learns patterns and predicts likely responses to prompts, unlike traditional AI that mainly classifies or predicts from historical data. Teams discover models in Vertex AI Model Garden (opens in a new tab) and access them as managed APIs or self-deployed endpoints.
A modality describes the kind of data a model can accept or generate, and one model can span several modalities. Text models use large language models (LLMs) to understand context and generate responses, code models turn natural language prompts into functions, tests, and scripts, image models such as Imagen learn visual patterns through diffusion, and multimodal models such as Gemini handle combinations of text, images, video, and audio. Grouping by modality helps match the model to the task inputs and outputs the business needs.
Three shared mechanisms support all modalities. Embeddings convert text, images, and video into arrays of floating point numbers called vectors, where closer distance in vector space means similar meaning, and retrieval-augmented generation (RAG) uses them to match a query to relevant documents. Grounding connects outputs to verifiable sources such as company documentation to reduce hallucinations. Function calling connects LLMs to external tools: the model analyzes the prompt, generates structured data naming the function and parameters, the function runs, and results return to the model for its final answer.
Vertex AI Model Garden is the central library where teams list available models, review verified deployment settings, and deploy open models to endpoints for prediction. Deployment follows a fixed order: first list deployable models and record the model ID, then view the verified machine type, accelerator type, and container image, and only after that deploy to an endpoint and send prediction requests. Deployment consumes quota in a designated quota project, so the project must be set before listing and deploying. Governance surrounds this flow with IAM permissions that define who can do what, quota limits that control usage, and evaluation metrics such as ROUGE for summarization that compare responses to reference answers.
Model choices in Model Garden balance who maintains the model and how much work the team accepts:
| Model group | Examples | How teams use it |
|---|---|---|
| First-party Google | Gemini models, Imagen | Call a managed API without managing servers; choose batch or online inference |
| Third-party partner | Anthropic Claude | Call through a partner SDK on Vertex AI and extend with tool calls |
| Open-source | Gemma, Llama, Hugging Face models | Self-deploy to a Vertex AI endpoint with chosen machine, accelerator, and license acceptance |
First-party models minimize infrastructure work and offer tuning such as Imagen style models for a consistent look. Partner models add specialized capabilities under partner terms, such as Claude calling a Places API for grounded answers. Open models maximize control over endpoint shape and container but require the team to operate the endpoint and accept the license.
Model selection is the process of balancing computational cost, response latency, and output quality for the task. In BigQuery (opens in a new tab), natural language inputs flow from SQL functions to pre-trained Gemini models and results flow back into tables for analysis. Larger models and longer prompts generally improve quality but increase token usage, cost, and latency, so teams match the function tier to the size and importance of the work.
The three BigQuery AI function tiers set the default trade-off. General-purpose functions such as AI.GENERATE and AI.EMBED give full control over model, prompt, and parameters for flexible inference and embeddings. Managed functions such as AI.CLASSIFY, AI.SCORE, and AI.AGG use streamlined syntax optimized for cost and quality over millions or billions of rows. Task-specific functions provide fixed behavior for translation, vision, speech, and document processing. Teams can estimate cost with AI.COUNT_TOKENS before running, start on a small sample to check quality, then switch to optimized managed mode for bulk execution.
Exam tip: Choose managed AI functions when the priority is optimized cost and quality over millions of rows, and choose general-purpose functions when the priority is control over the model, prompt, and parameters.
Continue with this study guide: Models
Platforms are the unified layer that moves work from model selection to a deployed application with repeatable pipelines and governance. Vertex AI (opens in a new tab) provides builder interfaces, orchestration frameworks, and serving services so prototypes become production systems. Grounding connects those systems to enterprise and real-world data so answers stay accurate and relevant.
The generative AI development flow is the ordered lifecycle that carries an idea to a monitored production service. Data and decisions flow forward from selection to deployment, with early stages repeating in an evaluation loop until quality is good enough:
Generative AI Studio is most used in the prompt engineering stage for Gemini and PaLM 2 models, but its outputs feed tuning and deployment. A marketing prototype shows the full chain: user data flows from sources into BigQuery, a Dataflow pipeline derives insights, Eventarc triggers a Cloud Run service, that service sends insights with a prompt to the Gemini Enterprise Agent Platform, the model generates audio, video, and text, and Cloud Run uploads the assets to Cloud Storage for the web portal to display.
Agent Builder and orchestration turn chosen models into deployable AI agents and manage them through operation. Gemini Enterprise Agent Platform Vision shows the pattern for video solutions: video flows from cameras into ingestion infrastructure, then flows from ingestion into an orchestration framework that routes it to general, custom, and specialized models, and analyzed output flows from the models to Vision Warehouse or BigQuery for querying. Vertex AI Pipelines automates, monitors, and governs ML systems, Vertex AI Inference serves predictions from trained models, Feature Store manages features for online serving, and Model Monitoring watches for skew and drift.
Infrastructure for orchestration depends on control needs. AI and ML orchestration on Cloud Run runs workloads directly on scalable infrastructure without managing Kubernetes clusters. AI and ML orchestration on GKE orchestrates the complete lifecycle with Kubernetes control for complex container-based training and serving. Grounding improves factual accuracy by pulling context at query time from vector stores such as Cloud SQL and AlloyDB for PostgreSQL, knowledge graphs that link connected facts, and real-world services such as Google Maps Platform and Google Calendar. In multi-agent designs, the app can be publicly accessible on one side of a network boundary while private connections carry agent-to-agent and agent-to-tool traffic on the other side.
Exam tip: Pipelines automates and governs the ML system, while Inference serves predictions from an already trained model.
Continue with this study guide: Platforms
Agents are the layer where AI shifts from answering questions to carrying out multi-step business processes with little human help. An AI agent is a system built on an LLM that interprets a goal, makes a plan, and executes steps while tracking context. Data flows from business systems to the agent for context, and actions flow from the agent back to those systems to complete tasks.
Static retrieval finds or creates information but stops at an answer, while an agent adds goal-directed execution through tools and business systems. The LLM acts as the reasoning core that understands language, weighs options, and decides the next step. Planning breaks the goal into ordered sub-tasks and revises the order as results return, and memory carries context from earlier messages and intermediate results to later decisions so the agent stays consistent. Short-term memory holds the current task state, while longer-term memory can retain preferences and key facts across sessions.
A typical autonomous workflow follows a fixed order where the agent owns the middle steps and the business controls boundaries:
This pattern shortens cycle times, reduces manual handoffs, applies the same rules consistently, and lets operations scale without adding staff at the same rate. New always-on and personalized services become possible, but only with clear limits on data access and escalation paths so errors do not spread.
Exam tip: Static generative AI retrieves or generates information; an AI agent goes further by planning and executing actions to complete a multi-step task.
Tool integration lets an agent reach beyond its model to query databases, call APIs, and use verified business context. In BigQuery, the agent translates a user question into SQL that calls the right AI or ML function, and a verified query prepared by a data scientist can embed those functions for recurring reports. Available functions include AI.SIMILARITY for similar content, AI.SEARCH for semantic search, AI.AGG for AI-assisted summarization, AI.PREDICT for predictions, and ML functions for trends, seasonality, change points, and correlation. The choice between search functions depends on table configuration: the agent uses AI.SEARCH only on tables with autonomous embedding generation enabled, and on any other table it falls back to AI.SIMILARITY, which generates embeddings in real time.
Knowledge Catalog is the semantic layer that gives the agent the business meaning behind the data. It connects to warehouses such as BigQuery, operational databases, unstructured stores such as Cloud Storage (opens in a new tab), and runtime metastores, then exposes grounded schemas to models and applications. It distinguishes semantics such as mapping txn_qty to purchased quantity, structured data in defined schemas, and unstructured data such as PDFs and images that are parsed into a graph profile of nodes and edges. Context flows in order from source systems into Knowledge Catalog for mapping, from Knowledge Catalog to the agent through MCP connectors or LookupContext APIs, and from orchestration frameworks such as Agent Development Kit (ADK) or LangChain into the LLM prompt before the model chooses a function.
Continue with this study guide: Agents
Applications are the top layer of the stack that faces employees and customers directly in natural language. Users do not build models or write code; they ask questions, receive grounded answers, and complete work inside familiar tools. Business value comes from grounding answers in enterprise data, which turns a general model into a trusted tool for support, search, and workflow automation.
Retrieval-augmented generation (RAG) is the method managed support applications use to ground answers in a knowledge base before generating a response. In the Google Cloud example, containerized services run in a Google Kubernetes Engine (GKE) (opens in a new tab) cluster, the support knowledge base lives in Cloud Storage as the source of truth, and the Gemini API performs retrieval and generation. Generation happens only after retrieval, so the final answer depends on enterprise sources rather than on the model alone. Because the application sits between users and enterprise knowledge, it improves response speed and satisfaction without requiring users to understand the models.
The grounded support flow runs in a fixed order:
Exam tip: A grounded support answer is retrieved first from Cloud Storage and generated second by Gemini — retrieval before generation is what makes the insight data-driven.
Horizontal and vertical applications differ by business scope, not by AI technique, and both can combine generative and traditional AI. Horizontal applications target general functions shared across industries, such as content creation, conversational search, document understanding, and code generation. Vertical applications are tailored to a specific sector such as media, manufacturing, cybersecurity, or ecommerce and depend on industry data, workflows, and compliance rules. A personalized marketing example shows a vertical flow: user data flows into BigQuery (opens in a new tab), a Dataflow pipeline derives demographic and purchasing insights, Eventarc triggers a Cloud Run service, that service sends insights to Gemini Enterprise Agent Platform to generate per-user assets, and the portal retrieves the assets from Cloud Storage.
| Dimension | Horizontal AI applications | Vertical AI applications |
|---|---|---|
| Scope | General functions across many industries | Specific sector or domain |
| Examples | Chatbots, support search, summarization, code generation | Personalized marketing, defect detection, threat analysis |
| Strategic advantage | Broad reuse, faster go-to-market, lower operational complexity | Deeper domain fit and compliance alignment |
| Key needs | May need prompt engineering or tuning for specialties | Requires industry data, domain experts, and process change |
Business value is defined before workflows change by working backward from measurable goals. Leaders name the goal and success criteria, decide whether generative AI, traditional AI, both, or no AI fits, identify end users and interaction expectations, link the choice to efficiency and experience, and identify required process changes. Traditional AI classifies or predicts from structured historical data, such as forecasting churn or segmenting customers, while generative AI creates summaries and new content and can explain those predictions in natural language. Success is tracked through financial gains, operational efficiency such as faster resolution, and customer experience such as satisfaction and retention.
Continue with this study guide: Applications
Single-host Tensor Processing Units (TPUs) in a managed instance group (MIG) each handle their own separate independent workload, while a multi-host TPU slice is a group of interconnected TPU virtual machines that work together as one unit on a single workload. Adding single-host virtual machines adds more independent capacity for inference without dependence between virtual machines, whereas in a multi-host slice data and computation flow across the connected virtual machines and they depend on each other to finish.
Google Kubernetes Engine (GKE) is for large-scale training and inference and custom high-performance platforms with deep control and portability, Vertex AI is for rapid development and fast time-to-value without managing infrastructure, and Cloud Run is for event-driven apps and smaller models with scale-to-zero serving. GKE requires Kubernetes expertise but controls cost at scale with Spot VMs and Flex-start VMs and serves low-latency inference with model-aware routing through Inference Gateway, while Cloud Run scales instances automatically on demand and Vertex AI offloads setup and operations to shorten time-to-market.
In Vertex AI Model Garden, first-party Google models are called as a managed API without managing servers, third-party partner models are called through a partner SDK on Vertex AI, and open-source models must be self-deployed to a Vertex AI endpoint. Examples include Gemini models and Imagen for first-party use, Anthropic Claude for partner use, and Gemma, Llama, and Hugging Face models for open-source use, where open models require choosing the machine type, accelerator, and license acceptance and operating the endpoint.
Retrieval-augmented generation (RAG) retrieves relevant resources from Cloud Storage first and generates a grounded answer with Gemini second. The support desk passes the customer question to a knowledge retriever that asks the Gemini API to identify relevant resources, fetches those resources from Cloud Storage, and hands the question plus resources to a solution generator that asks the Gemini API to generate the detailed solution for delivery to the customer.
Last updated