Prepare and test your skills
Prepare and test your skills
Full-stack infrastructure and hardware optimization means the compute, networking, storage, and software layers are designed and tuned together to run generative AI at scale. Data sits at the heart of all AI initiatives because training a deep learning model requires collecting and storing a large amount of data. With accelerators such as graphics processing units (GPUs) and Cloud TPUs, the speed of moving data from storage to the training process becomes increasingly important for both performance and cost-efficiency.
Purpose-built accelerators are specialized chips that perform AI math much faster than general CPUs, so training and inference finish sooner and cost less per workload. Data flows from cloud storage to the accelerator attached to the virtual machine (VM), where the training process consumes it, and the trained model then serves inference requests. Google Cloud offers Compute Engine accelerator-optimized machines where the GPU model is automatically attached to the instance, plus Cloud TPUs designed specifically for large-scale AI.
| Machine family | GPU attachment | Choose when |
|---|---|---|
| A-series | GPU automatically attached, including NVIDIA GB300 Ultra, GB200, B200, H200, H100, and A100 models | Large-scale model training that needs maximum accelerator power and scale |
| G-series | GPU automatically attached, including NVIDIA RTX PRO 6000 and NVIDIA L4 models | Training smaller models and single-host inference |
| N1 general-purpose | Attach a select GPU such as NVIDIA T4, P4, or V100, except on shared-core f1-micro and g1-small | Flexible general workloads that need an added GPU |
GPUs attached to Spot VMs work like normal GPUs but persist only for the life of the VM, and they are available at lower spot prices. Choose Spot VMs when the workload can tolerate interruption in exchange for lower cost.
Exam tip: Choose A-series machines for large-scale training and G-series machines for smaller-model training and single-host inference; both have GPUs automatically attached, while N1 requires you to attach a supported GPU.
Global networking and high-performance storage provide the fast paths that feed accelerators and connect distributed training nodes. The network carries data between storage buckets, accelerator hosts, and regions, while storage services hold training datasets close to compute so accelerators do not sit idle waiting for data. Because generative AI workloads scale across many hosts, this combined networking and storage layer determines how quickly training starts, how well it scales, and how efficiently expensive accelerators are used.
AI Hypercomputer (opens in a new tab) is Google Cloud's performance-optimized infrastructure that combines accelerators, networking, storage, consumption models, and cluster management into one system for AI workloads. Cluster management controls how accelerator nodes are grouped and scheduled, and consumption models control how capacity is purchased and used, so teams can scale training and inference without managing each layer separately. The software stack on top includes TensorFlow Enterprise (opens in a new tab), which is enterprise-grade support and performance for TensorFlow from the creators of TensorFlow.
TensorFlow Enterprise makes accessing data on Google Cloud faster and easier for AI-enabled businesses. TensorFlow, open sourced by Google in 2015, runs for every user, but deploying it on Google Cloud adds optimized data access and supported performance. Together, the Hypercomputer infrastructure and the optimized TensorFlow stack reduce data bottlenecks from storage to accelerator and keep large generative AI workloads both fast and cost-efficient.
Synergy between data analytics and AI platforms means data management and model development work as one lifecycle instead of separate systems. Data analytics platforms store, organize, and query an organization's data, while AI platforms build and serve machine learning models. When these two layers are integrated on Google Cloud, data moves from storage into models and model outputs move back into the tools people already use, without teams copying data between separate systems.
BigQuery is the data warehouse that supplies feature data to Vertex AI Feature Store, a managed service for storing, serving, and managing machine learning features. A feature registry (opens in a new tab) is the central interface for recording the feature data sources that will be served for online inference. A feature group is a feature registry resource that maps to a BigQuery source table or view containing feature data, and a feature view is a logical collection of features materialized from that BigQuery source into an online store instance.
Feature serving is the process of exporting or fetching feature values for training or inference, and Vertex AI splits it into two modes. Online serving retrieves the latest feature values for a subset of the feature data source for online inferences, while offline or batch serving exports high volumes of feature data, including historical data, for offline processing such as model training. A feature record aggregates all feature values that describe one unique entity at a specific point in time, and a feature timestamp records when those values were generated. The feature view stores the customer's feature data and refreshes it periodically from the BigQuery source, so the AI platform reads current values rather than a stale export.
Exam tip: Online feature serving returns the latest values for real-time inference; offline (batch) feature serving exports historical values for model training.
Grounding is the practice of connecting a generative AI application to real-time data so its responses are informed and accurate, and the technique is called retrieval-augmented generation (RAG). Gemini models can evaluate whether the model's own knowledge is sufficient or whether grounding with Google Search is required for up-to-date internet content. Grounding can be implemented with an index of data in a search engine, and many search engines store embeddings in a vector database, which is an optimal format for operations like similarity search.
A vector is a numerical representation of text, images, or video that captures relationships between inputs, and embedding models such as text-embedding translate text prompts into these vectors so vector functions can compare and sort text by relative semantic distance. Google Cloud offers several grounding paths, and the choice depends on how much control and management the team wants:
| Option | What it is | Choose when |
|---|---|---|
| Vertex AI Search (being renamed Agent Search) | Fully managed optimized solution supporting most data sources | You want managed grounding that prevents direct access to the underlying embeddings |
| Vertex AI RAG Engine | Managed orchestrator with a LlamaIndex-like interface | You are building a search engine for RAG and want a managed orchestrator |
| Vertex AI Vector Search | Specialized vector database | You need low-latency vector search, large-scale serving, or an optimized vector database |
| Cloud SQL or Firestore | Existing non-vector database | Your data is accessed programmatically (OLTP) and you can use a query result in the model prompt |
Cloud SQL (opens in a new tab) for PostgreSQL version 12 and later can integrate with Vertex AI so that large language models hosted in Vertex AI are applied directly to database data. The integration extends SQL syntax with two functions: invoking predictions calls a model using SQL within a transaction, and generating embeddings has an embedding model translate text prompts into numerical vectors that can then be used as inputs to vector functions. Because predictions happen inside the database, teams can make real-time predictions and gain insights without moving data out to a separate system.
Enabling the Cloud SQL integration requires several conditions to be met in order. The requirements include a dedicated core machine type, maintenance version R20240130 or later, the cloudsql.enable_google_ml_integration database flag turned on, the Vertex AI API enabled, and the Cloud SQL service account holding the aiplatform.endpoints.predict permission. The Cloud SQL instance and the Vertex AI endpoint must also be in the same region, because the instance sends its request to the endpoint in that region. Two limits apply to the traffic: a single request cannot exceed the gRPC limit of 4 MB, and requests are capped at 1,500 per minute per region per model per project.
Vertex AI Model Registry is a central repository where the lifecycle of machine learning models is managed, giving an overview of models so they can be organized, tracked, and versioned. When a model version is ready to deploy, it can be assigned to an endpoint directly from the registry or deployed using aliases. Supporting services record what happened along the way and help improve the model:
data governance (opens in a new tab) across the lifecycle protects customer data while models are built, deployed, and served. Google's AI/ML Privacy Commitment states that customers should have the highest level of security and control over data stored in the cloud, and that commitment extends to Google Cloud generative AI products. Google enforces it through data governance practices that include reviews of the data used in product development, with further detail in the Customer Data Processing Addendum (CDPA).
For Agent Search, customer data is not used to train foundation models; the foundation models are frozen and only process input to provide output for the service. When customer-specific models apply, they are built automatically and used only in the context of the customer's own project, and any customer data used to create them is likewise not used to train foundation models.
Integration across Workspace and consumer applications means embedding generative AI capabilities directly into the tools people already use. Email, documents, search, and chat gain AI help inside the normal workflow instead of requiring a switch to a separate product. The AI draws on the same data those tools already hold, such as emails, chat conversations, product documentation, and websites, to produce responses that are specific, reasoned, and actionable.
Foundation models (opens in a new tab) are large, powerful models trained on vast amounts of data across multiple modalities like text, images, video, and audio. Foundation models learn patterns from their training data and use statistical modeling to predict likely responses to prompts and generate new content. Google offers a variety of generative AI foundation models accessible through a managed API, and they are reached through the Vertex AI model garden.
An embedding is a numerical representation of data — text, images, or video — that captures relationships between different inputs. Embeddings are generated during training by converting content into arrays of floating-point numbers called vectors, which map data into a continuous vector space where closer proximity reflects similar meaning. This lets models recognize nuanced patterns and relationships that raw data would hide. Systems that use retrieval-augmented generation (RAG) (opens in a new tab) rely on embeddings to match a user query with relevant knowledge: the query becomes an embedding, it is compared against the embeddings of documents in a knowledge base, and a similarity search retrieves the most semantically relevant information.
A seven-step process flow showing enterprise data converted to embeddings, stored in a vector database, searched by similarity, then added to a prompt for the Gemini model to produce a grounded response.
Function calling connects large language models (LLMs) to external tools like APIs and functions so responses can go beyond static knowledge. When a user prompts the model, the model analyzes the request and generates structured data that specifies which function to call and what parameter values to use. The function then runs, returns its results to the model, and the model folds those results into its reasoning to produce a response. This gives the model access to real-time information from databases, customer relationship management systems, and document repositories.
Native AI features in everyday applications interpret complex language and nuances rather than focusing only on specific keywords, which is how a traditional AI chatbot works. This lets generative AI understand the full context of a request and respond to conversational queries naturally. It can also summarize complex information drawn from many sources, so a user does not have to read through every email, document, or chat thread to find the answer.
Generative AI chatbots give users the flexibility to express themselves in their preferred method — text, voice, or image — and the chatbot uses all of that input together to improve resolution. For example, a customer can share a photo of a damaged product during a chat, and generative AI combines the customer's description with the photo to improve diagnostic and troubleshooting accuracy.
Exam tip: A traditional AI chatbot focuses on specific keywords, while generative AI interprets the full context and nuances of a request — a distinction that matters when comparing older support tools with generative AI-powered ones.
The automated support workflow is the ordered interaction between a generative AI-powered chatbot and a support ticketing system. First, the chatbot creates and populates the ticket by interfacing directly with the support ticketing system and filling in customer details, technical issue categorization and priority, a full transcript of the conversation for context, and a summarization of the core issue. Then, for new and complex issues, the chatbot assigns the ticket to the correct team along with supporting context such as a summary of the issue and the conversation. The chatbot also uses generative AI to judge the urgency, sentiment, and complexity of the issue so tickets are prioritized effectively.
Compute Engine A-series machines suit large-scale model training that needs maximum accelerator power and scale, while G-series machines suit training smaller models and single-host inference. Both A-series and G-series have graphics processing units (GPUs) automatically attached, while N1 general-purpose machines require you to attach a supported GPU such as NVIDIA T4, P4, or V100.
AI Hypercomputer is Google Cloud's performance-optimized infrastructure that combines accelerators, networking, storage, consumption models, and cluster management into one system for AI workloads. Cluster management controls how accelerator nodes are grouped and scheduled and consumption models control how capacity is purchased and used, so teams can scale training and inference without managing each layer separately.
Online feature serving in Vertex AI Feature Store retrieves the latest feature values for a subset of the feature data source for online inferences, while offline or batch serving exports high volumes of feature data including historical data for offline processing such as model training. The feature view stores the customer feature data and refreshes it periodically from the BigQuery source, so the AI platform reads current values rather than a stale export.
Cloud SQL for PostgreSQL version 12 and later can apply Vertex AI models directly to database data without moving data out to a separate system. The Cloud SQL instance and the Vertex AI endpoint must be in the same region, and traffic is limited to 4 MB per single request and 1,500 requests per minute per region per model per project.
A corporate communications team wants to improve efficiency and reduce the time employees spend drafting press releases, summarizing lengthy email threads, and retrieving relevant organizational insights. The leadership team wants an AI solution that works directly inside the tools employees already use daily—such as their email client, word processor, and search interfaces—without requiring staff to switch between external standalone AI applications or write custom code.
Which approach best leverages Google's integrated AI ecosystem to streamline these user workflows?
Implementing a traditional rule-based chatbot that relies on keyword matching to process and draft communications
Deploying custom containerized microservices on Google Kubernetes Engine to host open-source models for basic office tasks
Adopting native generative AI integrations within Google Workspace applications like Gmail and Docs
Requiring employees to manually copy text into a standalone external web chatbot and paste the generated results back into their documents
Last updated