Prepare and test your skills
Prepare and test your skills
Foundation model limitations are predictable weaknesses that come from training on historical data and generating answers from patterns. Large language models (LLMs) (opens in a new tab) only know what was in their training data up to a fixed knowledge cutoff, so they hold a static snapshot with no awareness of later events. Data dependency means output quality depends on both that historical training data and the prompt data supplied for each request. When training data contains errors, gaps, or bias, the model repeats those flaws, and when prompt data is inaccurate, the response built from that prompt can also be false or poor.
Hallucinations and edge case failures happen because models learn language patterns rather than storing verified facts. A hallucination is a fluent, confident response that is not factually grounded, while an edge case failure is a wrong outcome on a rare or complex input that was not well represented in training. Bias is the tendency to reproduce prejudices from training data, while fairness is the goal of giving comparable quality to all users and groups. The chain runs from data to model to user: distorted training patterns flow into the model, the model applies those patterns, and the user receives output that can reinforce stereotypes, give superficial answers on specialized topics, or provide weaker quality for non-English languages and dialects.
| Limitation | What causes it | How it shows up in output |
|---|---|---|
| Bias amplification | Existing prejudices in training data | Outputs that reinforce unequal treatment |
| Language quality gaps | Underrepresentation of dialects and non-English languages | Inconsistent quality for less-represented languages |
| Limited fairness coverage | Benchmarks cover mainly gender, race, ethnicity, and religion in English | Risks outside those groups and languages are missed |
| Edge cases | Rare situations missing from training | Overconfidence or misinterpreted context |
Grounding is the process of connecting model output to specific, verifiable sources at request time instead of relying only on training patterns. Generative AI applications (opens in a new tab) use grounding to reduce hallucinations by sending verified facts from a source to the model, so the model generates its answer from retrieved evidence. Retrieval-augmented generation (RAG) retrieves semantically relevant documents from a knowledge base and gives them to the model before generation. Function calling connects the model to live tools in a different way by letting the model request data or actions from databases, customer relationship systems, and document repositories.
RAG depends on embeddings and vector search to match meaning rather than exact words. Embeddings are numerical representations of text, images, or video that capture relationships between inputs, stored as arrays of floating point numbers called vectors. The ordered RAG flow runs from user to knowledge base to model and back to the user:
A process flow showing the four ordered steps of retrieval-augmented generation: the user query is converted into an embedding, compared against document embeddings in the knowledge base via similarity search, the most relevant passages are retrieved, and the model generates a grounded response returned to the user.
| Approach | What it supplies | Choose when |
|---|---|---|
| RAG with embeddings | Relevant documents from a knowledge base | The answer should be based on stored content |
| Function calling | Real-time data and actions from external tools | The answer needs live information or an action |
| Grounding with Google Search | Up-to-date information from the internet | The model must decide if its own knowledge is sufficient |
Exam tip: Training data determines what the model already knows, while grounding context supplied with the prompt determines what current or specific facts it can use accurately.
Prompt engineering is the practice of designing and improving inputs to guide a foundation model without changing model weights. Clear instructions, context, and examples flow from the prompt to the model, and the refined response flows back to the user. Because weights do not change, teams can test revisions quickly with minimal compute, data, and cost. A prompt can define a role, specify output format, add domain context, and show examples.
Model fine-tuning is the process of further training a pre-trained foundation model on a focused dataset to specialize it for a task, tone, or vocabulary. Training data flows from the curated dataset to the base model, and the result is an adapted model that responds more consistently without long instructions every time. Fine-tuning requires more upfront investment in quality data, compute, and evaluation. The two methods form one lifecycle: first prompts are refined to define desired behavior with minimal spend, then only after demand proves steady and quality gaps remain does the team invest in fine-tuning.
| Method | Resource needs | Choose when |
|---|---|---|
| Prompt engineering | Low compute and data needs, fast iteration | Requirements change often or resources are limited |
| Model fine-tuning | Higher compute, curated data, and evaluation effort | The task is specialized, repeated at scale, and needs consistent tone |
Continuous monitoring and evaluation keeps a deployed model reliable, secure, and aligned with business goals after release. Model Monitoring on Agent Platform tracks input data and predictions for shifts, while the Gen AI evaluation service tracks output quality and safety. Insights flow from monitoring into evaluation and retraining, so reliability, accuracy, and fairness are maintained. When metrics cross thresholds, alerts trigger actions such as retraining or rollback to a stable version in Model Registry.
Key performance indicators (KPIs) are business outcomes such as revenue, satisfaction, or task completion rate, supported by service-level objectives (SLOs) and the four golden signals:
Concrete SLO targets make these signals actionable, such as 99.9% of API calls must return success and 95th percentile inference latency must be below 300 ms. Drift monitoring compares live behavior to a baseline built from a golden dataset, where training-serving skew compares live inputs to training data and prediction drift tracks shifts in the model’s own outputs. Vertex AI lifecycle management tracks tuning jobs that adjust models, evaluations that measure performance, and inference that serves results as batch inference for groups of requests in one file or online inference for real-time individual requests. Human-in-the-loop (HITL) places a person inside the process to review output before use, and human supervision is specifically required where decisions could materially impact individual rights.
Prompt engineering is the iterative process of designing, testing, and refining inputs to get specific, high-quality results from a large language model. A large language model (LLM) is a foundation model trained on vast text to predict what is likely to come next, and the prompt is the main control over that prediction. Data flows in one direction from user goal to prompt wording to model response. When the prompt states the task, context, audience, and desired format, the model does not have to guess missing details and the output aligns more closely with intent.
Careful prompt design also reduces hallucinations and supports safety by keeping the model grounded. Hallucinations are confident but false statements that become more likely when a prompt is vague or asks for information the model does not have. Teams reduce guessing by providing source material for the answer, asking for specific scope and format, and instructing the model to say it does not know when uncertain. Better organization also improves efficiency because structure uses ordering, labeling, and delimiters to help the model parse instructions, which reduces retries, token use, and latency.
In enterprise summarization tools, the structured input is called a preamble, which combines a task description that sets role and goal with additional instructions that control execution. A complete preamble combines answer style for tone and audience, output format such as bullets or tables, length limits such as concise or under 200 words, required details that must not be omitted, prohibited topics with polite refusal rules, and verifiability rules that keep claims within given context. Effective teams refine preambles through iterative testing, user feedback, staying updated as models change, and evaluation across use cases to catch bias.
Prompting strategies (opens in a new tab) differ mainly in how many input-output examples are included inside the prompt for in-context learning. Zero-shot prompting, also called direct prompting, provides no examples and relies solely on pre-existing knowledge from training. One-shot prompting provides one example to show the desired pattern, format, or style. Few-shot prompting provides several example pairs so the model can better recognize the pattern and apply it to a new request.
| Strategy | Examples in prompt | Choose when |
|---|---|---|
| Zero-shot | None | The task is straightforward and training knowledge is enough |
| One-shot | One input-output pair | The task needs a single clear example of format or tone |
| Few-shot | Several input-output pairs | The task is complex and needs stronger guidance |
Few-shot prompting provides the strongest in-prompt guidance because the model sees the pattern repeated, but all example strategies help control formatting, phrasing, scope, and overall patterns. Teams must test the number of examples because too few under-guides the model while too many can cause overfitting to the examples. Trying prompt improvement first can avoid moving to supervised tuning, reinforcement learning from human feedback tuning, model distillation, or a larger model, which often brings higher latency and higher cost measured by input and output tokens or node hours.
Role prompting controls output by telling the model who to act as and what boundaries to respect. The prompt opens with a role definition before the request, such as you are a polite customer service agent or you are an expert at Google Search. That persona tells the model which knowledge and style to use, so an agent sounds reassuring while a search expert focuses on concise terms. Persona constraints then create contextual boundaries through explicit limits, such as only answering about the provided menu, politely declining unrelated questions, staying polite about operator requests, and avoiding extra behavior like unneeded follow-up questions.
Prompt chaining breaks complex work into linked steps where each output becomes the next input. Each step handles a narrow piece so the model reasons over already-processed material rather than the raw task, which improves accuracy and keeps reasoning on track:
$information or $last-user-utterance.Chains can also pull in current data between prompts because data flows from an outside system into a parameter and then into a placeholder. Function calling connects the LLM to APIs where the model generates structured data naming the function and parameters, the function runs and returns results, and the model builds its response from those results. Dynamic retrieval connects the generator to a database or search engine where retrieved information is saved into a parameter when it is too large to paste directly.
Exam tip: When the answer must come only from provided information, include an explicit decline rule; without it the model falls back to training knowledge, which may not be true or up to date.
Chain-of-Thought (CoT) prompting improves logic by showing step-by-step reasoning before the answer. The prompt provides instructions and examples that demonstrate reasoning first, and the model follows that pattern by generating its own steps before producing structured output. Because models predict what is likely to come next, seeing and generating steps in order keeps the final answer grounded in prior logic. Order matters: first the step-by-step reasoning, then the final answer that depends on it.
The ReAct framework combines reasoning and acting by interleaving thought with concrete tool use. ReAct stands for Reasoning and Acting, and it lets the model think through a problem and then request help from another system that runs in its own environment. The result returns across the trust boundary into model context, where the model handles language and decisions while the tool handles factual lookup and real actions. The ReAct loop repeats in order:
Choose ReAct when the answer depends on changing data, lightweight tool calling, or dependent lookups, and avoid it when instructions alone are enough because extra calls add time and complexity. Common tools include search engines for current facts, databases for stored records, calculators for precise math, and APIs for actions like checking inventory. Thinking (opens in a new tab) as a built-in model capability that generates internal reasoning is distinct from explicit CoT instructions written in the prompt, so both versions should be tested and the more accurate one kept.
Vertex AI Studio (opens in a new tab) tools and parameters help writers create, refine, and compare prompts while controlling randomness and cost. The built-in AI assistant speeds up writing through Refine prompt with feedback such as shorter or more professional, and Help me write to describe the goal and generate a draft. The Compare feature shows prompts and responses side by side for a new prompt, a saved prompt, or a ground truth preferred answer, but it does not support media or multi-exchange chat prompts. Technique selection follows a test-driven workflow that defines the objective, tests systematically, and splits prompts that ask for too many distinct cognitive actions.
| Parameter | What it controls | How to choose |
|---|---|---|
| Temperature | Randomness in token selection | Lower for true or correct responses; higher for diverse results |
| Output token limit | Maximum text output, where a token is about four characters | Raise it when the answer needs to be longer |
| Thinking budget | How much the model thinks before answering | Auto for only when needed; Manual to set budget; Off to disable |
| Structured output | Forces machine-readable format such as JSON | Choose when output must be machine-readable |
| Grounding | Connects the model to Google Search, Google Maps, or your own data | Choose Google for public sources or Your data for RAG |
Max responses sets the maximum number of responses per prompt, while Top-P changes token selection. Structured output turns off grounding options, so structured output and grounding cannot be combined. Safety filters can still block responses even when parameters are set well.
Grounding (opens in a new tab) is the ability to connect model output to verifiable sources of information at generation time. A large language model without grounding must rely only on patterns learned during training, which can be incomplete or out of date. Grounding changes what the model is allowed to base its answer on without changing the model itself, so the response reflects retrieved facts rather than guesses. A response that cannot be traced to those sources is not considered grounded, and a claim that is only partially supported still counts as ungrounded.
How grounding connects models to data sources is a fixed retrieval flow across a trust boundary. The data source sits on one side of the boundary and the model sits on the other, with retrieved snippets bridging the two:
Because retrieval happens first and generation happens second, the answer reflects what the source returned for that request. Grounding tethers output to chosen sources, reduces hallucinations, and provides auditability through support links to sources, which matters most for customer answers, business decisions, and regulated work.
First-party enterprise data, third-party data, and world data differ by who owns the data and how the model reaches it. First-party enterprise data is information the organization owns or controls, reached through Agent Search, RAG Engine, Elasticsearch, or a custom search API for retrieval-augmented generation. Third-party specialized data is external data from a specialized provider, such as Google Maps for geospatial context, Web Grounding for Enterprise for controlled web answers, and parallel web search for recent information. General world data is broad public knowledge reached through Google Search grounding.
| Data source category | Primary source | Unique value |
|---|---|---|
| First-party enterprise data | Organization-owned documents and indexes | Anchors responses to internal content and supports auditability |
| Third-party specialized data | External providers such as Google Maps | Domain-specific context, compliance controls, or recency |
| General world data | Public web and search knowledge | Broad coverage across many topics |
A retail assistant can use Google Maps data for location-aware questions, while a bank can use Web Grounding for Enterprise for controlled web answers. Privacy, residency, and endpoint choices determine whether a source fits a workload. Data at rest remains in the customer-selected location, while machine learning processing location depends on endpoint type: jurisdictional multi-region endpoints keep processing within the United States or European Union, locational endpoints keep processing within that region such as us-central1, and global endpoints may process data anywhere without residency guarantees.
Exam tip: A claim that is only partially supported by retrieved facts counts as ungrounded, even if part of the sentence is correct.
Retrieval-augmented generation (RAG) is a grounding technique where relevant external information is retrieved and added to the model prompt before generation. Data flows from the grounding source to the prompt and then to the model, so the model reasons over retrieved documents instead of relying only on internal patterns. Foundation models need retrieval because they predict likely responses from statistical patterns and can invent plausible details when facts are missing, out of date, or specific to one organization. Gemini models can evaluate whether internal knowledge is sufficient or whether grounding with Google Search is required.
RAG finds passages by matching meaning through embeddings and vector search. Embeddings map content into a continuous vector space where closer proximity means more similar meaning, and many search engines store them in a vector database optimized for similarity search. The typical RAG flow moves in fixed order from retrieval to augmentation to generation, where retrieved documents flow into the prompt and the model generates a response constrained by that evidence. Supported sources include Google Search for internet information, an index of organizational data, results from Cloud SQL or Firestore, and data from online service APIs.
RAG also adds citations and source transparency by carrying source locations into the final answer. A RAG corpus is the managed collection of files that supplies grounding material, and teams can upload, import, list, get details about, or delete files in it. The detailed flow runs from prompt received to knowledge base search to retrieval of matching chunks to building an optimized prompt to generating the final answer. Citation links each fact to a page, paragraph, table, or row, and clear parser instructions help by telling the model what to extract and what to avoid, such as opinions or inference.
Google Cloud grounding offerings are managed search, data, and web paths that supply verifiable context before generation. In pre-built retrieval design, Vertex AI Search first retrieves relevant passages from authoritative enterprise data, and then the model uses both the user question and those passages to generate a response. Vertex AI Search is a managed search engine, so Google Cloud operates the optimized search and vector infrastructure and prevents direct access to underlying embeddings. The trust boundary sits at the search-engine layer because applications query the search engine rather than reading raw vector data.
Vertex AI Search can return snippets, extractive answers, and extractive segments as verbatim extracts. Snippets are short previews with highlighting for judging relevance, while extractive answers and extractive segments are longer verbatim sections used when exact wording or broader context matters. The request controls output with returnSnippet for snippets and maxExtractiveAnswerCount for extractive answers, where the default is 0 and the maximum is 5. Live web grounding adds current facts because training data is a fixed snapshot, so Google Search results flow into the model as context and then into grounding metadata with title, link, and supporting text that the user can open to verify claims.
| Option | Management level | Choose when |
|---|---|---|
| Vertex AI Search | Fully managed search engine | You want RAG over many sources without managing embeddings |
| Vertex AI Vector Search | Managed vector database | You need low-latency search with custom tuning |
| Vertex AI RAG Engine | Managed orchestrator | You are building a search engine for RAG with orchestrator control |
| AlloyDB or Cloud SQL | SQL database | Data already lives in SQL or needs programmatic access |
| BigQuery | Analytical warehouse | Data is a large analytical dataset with SQL access |
Sampling parameters and settings control how random, large, and safe model behavior is. Temperature is the generation parameter that controls prediction randomness on a scale from 0.0 to 1.0, where lower values produce less random output and higher values produce more random output. Top-K sampling changes how the model selects tokens for output in the same direction, where lower values produce less random responses and higher values produce more random responses. Lower values support expected and repeatable output, while higher values support creative output but raise the chance of inaccurate results.
Output length and token limits cap how much text a model takes in and generates. Every request is measured in tokens, which are small pieces of text, and every model has a fixed maximum input and output token limit. Setting a maximum output length below that cap limits resource use, reduces waiting time, and keeps answers the right size, where a short cap fits summaries and a longer cap fits detailed drafts. Fixed limits are also a safety boundary: if input or output exceeds the limit, safety classifiers are not applied, which can lead to poor performance and unexpected results.
Safety services and incident response protect data by gating access, recording access, and correcting violations. Gemini Enterprise Agent Platform APIs provide safety attribute scoring to test filters and set confidence thresholds, and Vertex AI Studio includes built-in content filtering. Google recommends assessing security risks, performing safety testing, configuring safety filters if required, and then monitoring content with user feedback. Related controls include Access Approval that requires customer approval before administrator access, Access Transparency that logs that access, Assured Workloads for compliance packages, Cloud KMS with Cloud EKM for external keys, and Confidential Computing that encrypts data while it is being processed.
Continue with this study guide: 3.3 Identify grounding techniques and their use cases
Prompt engineering is the practice of designing and improving inputs to guide a foundation model without changing model weights, while model fine-tuning further trains a pre-trained foundation model on a focused dataset to specialize it for a task, tone, or vocabulary. Prompt engineering needs low compute and data and allows fast iteration, so it fits when requirements change often or resources are limited. Model fine-tuning requires higher compute, curated data, and evaluation effort, so it fits when the task is specialized, repeated at scale, and needs consistent tone.
Retrieval-Augmented Generation (RAG) retrieves semantically relevant documents from a knowledge base and gives them to the model before generation, while function calling lets the model request data or actions from external tools such as databases, customer relationship systems, and document repositories. Retrieval-Augmented Generation (RAG) depends on embeddings and vector search to match meaning rather than exact words through query embedding, similarity search, retrieval, and grounded response. Choose Retrieval-Augmented Generation (RAG) when the answer should be based on stored content and choose function calling when the answer needs live information or an action.
Zero-shot prompting provides no examples and relies solely on pre-existing knowledge from training, one-shot prompting provides one input-output example to show the desired pattern, format, or style, and few-shot prompting provides several example pairs for stronger guidance. Zero-shot prompting fits straightforward tasks where training knowledge is enough, one-shot prompting fits tasks needing a single clear example of format or tone, and few-shot prompting fits complex tasks needing stronger guidance. Too few examples can under-guide the model while too many can cause overfitting to the examples, so the number of examples must be tested.
Chain-of-Thought (CoT) prompting improves logic by showing step-by-step reasoning before the answer so the model generates its own steps before producing structured output, with order mattering because the final answer depends on prior logic. The ReAct (Reasoning and Acting) framework combines reasoning and acting by interleaving thought with concrete tool use through a Thought, Action, Observation, and repeat or finish loop where data flows from the external tool back to the model as new evidence. Choose the ReAct (Reasoning and Acting) framework when the answer depends on changing data, lightweight tool calling, or dependent lookups and avoid it when instructions alone are enough because extra calls add time and complexity.
Last updated