Prepare and test your skills
Prepare and test your skills
Apply Prompt Engineering and Model Fine-Tuning covers two practices for improving foundation model outputs that are inaccurate, generic, or unsafe. Prompt engineering (opens in a new tab) refines the input to get better output immediately without changing the model. Model fine-tuning further trains the base model on specialized data to adapt it for a specific task. The choice between them depends on workload requirements, available resources, and the level of performance and consistency needed.
Prompt engineering is the practice of designing and improving inputs to guide a foundation model toward more accurate and relevant responses. The flow moves from the user to the model to the user: clear instructions, context, and examples flow from the prompt to the model, and the refined response flows back to the user. Because the model weights do not change, teams can test and revise prompts quickly with minimal compute, data, and cost.
Prompt engineering improves factual accuracy and contextual relevance by adding direction the base model did not receive in original training. A prompt can define a role, specify the output format, add domain context, and show one or more examples of desired behavior. Each revision is evaluated against actual outputs, and the prompt is right-sized to match demand without becoming overly long or costly to run. When monitoring shows that prompt changes alone cannot deliver stable quality, a more specialized adaptation may be needed.
Model fine-tuning is the process of further training a pre-trained foundation model on a focused dataset to specialize it for a task, tone, or vocabulary. Training data flows from the curated dataset to the base model, and the result is an adapted model that responds more consistently in that specialty without needing long instructions every time. Fine-tuning requires more upfront investment, including quality data, compute for training, and evaluation to confirm improvement and safety.
Fine-tuning depends on workload requirements in the same way as choosing workload-specific cloud resources. A steady, mission-critical workload with strict quality needs justifies premium resources and deeper adaptation, while an occasional or exploratory workload fits cost-efficient and flexible resources. Fine-tuning is justified when volume is steady, language is specialized, and prompt engineering cannot meet quality needs reliably. After deployment, resource use and output quality should be tracked continuously so allocation stays aligned with actual demand.
Choosing between prompt engineering and fine-tuning means balancing cost and functionality against performance requirements and resource availability. Prompt engineering offers speed and efficiency with low commitment, while fine-tuning offers deeper adaptation with higher upfront cost and longer-term payoff.
| Method | Resource needs | Choose when |
|---|---|---|
| Prompt engineering | Low compute and data needs, fast iteration | The task needs immediate refinement, requirements change often, or resources are limited |
| Model fine-tuning | Higher compute, curated data, and evaluation effort | The task is specialized, repeated at scale, and needs consistent accuracy or tone |
The two methods can compose into one lifecycle. First, prompts are refined to define desired behavior and validate the use case with minimal spend. Then, only after demand proves steady and quality gaps remain, the team invests in fine-tuning to lock in specialized performance. This progression avoids overprovisioning and keeps resources aligned with dynamic workload needs.
Exam tip: Start with prompt engineering for fast refinement, and move to fine-tuning only when repeated testing shows prompts cannot deliver the required consistency for a specialized task.
Implement Grounding and Retrieval-Augmented Generation (RAG) explains how to anchor model responses in verifiable data to improve factual reliability. Grounding (opens in a new tab) is the process of connecting model output to specific, verifiable sources such as internal documentation, project data, or communication records. Retrieval-augmented generation (RAG) is a common grounding technique that retrieves relevant external information and gives it to the model before generation. This reduces hallucinations, where the model generates content that is not factual.
Grounding to reduce hallucinations means giving the model access to real sources instead of letting it answer from training data alone. The grounding source sends verified facts to the model, and the model generates a response based on those facts for the user. The retrieved information can come from a knowledge base that includes company documentation, project-specific data, or communication records. Grounded responses are more accurate, reliable, and useful because their statements can be traced to source data.
Embeddings are numerical representations that map data into a continuous vector space where closer proximity reflects similar meanings. Embeddings are created by converting text, images, and video into arrays of floating point numbers called vectors. Embeddings often reduce dimensionality, which improves computational efficiency and enables processing of large datasets. Large language models (LLMs) rely on embeddings to understand context and meaning, which helps them generate coherent and relevant responses.
RAG systems use embeddings to match user queries with relevant knowledge. When a query is posed, the query is converted into an embedding. That query embedding is compared to embeddings of documents in the knowledge base through similarity searches in the vector space. Retrieval is usually done using a search engine, which uses an index embedded with the semantic meanings of the source text. This lets the system retrieve the most semantically relevant information for generation.
The ordered RAG flow is the sequence that carries information from the user query to the knowledge base to the model and back to the user. Retrieval happens first, generation happens second, and a grounding check happens before the response is returned. The stages run in this fixed order:
Only the retrieved facts flow from the knowledge base to the model, and only the checked response flows from the model to the user.
Configuring grounding with RAG Engine (opens in a new tab) in Agent Studio connects an agent prompt to a corpus of grounded data. RAG Engine on Gemini Enterprise Agent Platform is the service for grounding responses using RAG. The configuration flows from the Agent Studio console to model settings to the selected corpus, and grounded responses then flow back to users.
The setup runs in this order:
Exam tip: The default Top-K Similarity value for RAG Engine grounding is 20.
Components that support the RAG lifecycle are Google Cloud services and APIs that implement retrieval, understanding, search, and verification. The Agent Search Builder API allows mix-and-match building, so a RAG solution can use any of these services or APIs:
These components depend on each other in operation: the parser prepares documents for understanding, Vector Search finds relevant embeddings, the grounding generation API combines retrieval with generation, and the check API verifies the final output.
Grounding agents with Knowledge Catalog uses a unified semantic context graph to supply safe, relevant data to AI applications. AI applications and business users query the catalog to access data safely, and grounding agents connect LLM-based apps and agents to that catalog. The Context API performs low-latency grounding payload requests, while semantic search finds the right assets using natural-language queries. Data packaging bundles collections of tables, licensing details, and SLAs into secure, self-service data packages.
Knowledge Catalog integrates with BigQuery, Looker, Lighthouse runtime catalog, Gemini for AI agents, and Gemini Enterprise Agent Platform to feed tools and context to agents. In this design, governed data flows from source systems to the catalog to the agent, so agents ground their answers in packaged, licensed data rather than ungoverned copies.
Integrate Human-in-the-Loop (HITL) Workflows places a person inside the AI process to review, validate, or correct output before it is used. Human-in-the-loop (HITL) is an added layer of oversight for cases where automated controls alone cannot guarantee correct or responsible results. Google's Secure AI Framework (SAIF) (opens in a new tab) describes this pattern directly: AI enhances security work such as identifying threats and automating detection, while humans stay in the loop for critical decisions.
Foundation model limitations that require human review are known weaknesses that prompt tuning alone cannot fully remove. These limitations include:
The limitation that most directly calls for HITL is limited domain expertise. For specialized, complex use cases, models should be tuned on domain-specific data, and there must be meaningful human supervision in contexts with the potential to materially impact individual rights.
The position of human review in the safety workflow is the final step in the recommended sequence for using generative AI safely and responsibly. Human judgment catches what configured filters and testing did not, and observations from review feed back into earlier safeguards. The sequence runs in this order:
Steps 1 through 3 build technical and procedural safeguards, and step 4 is where human judgment enters. Feedback and monitoring from step 4 inform whether earlier steps need to change, which makes the workflow a loop rather than a one-time setup.
Practices that implement human oversight combine feedback channels, monitoring, and governance controls that assign responsibility to identifiable people. These practices support fairness, interpretability, privacy, and security:
Customers remain responsible for adhering to Google Cloud's Acceptable Use Policy (AUP), the Generative AI Prohibited Use Policy, and any other Service Specific Terms or relevant requirements for AI and ML and generative AI services.
Human review alongside automated safeguards means each covers a different kind of risk. Vertex AI Studio provides built-in content filtering, and the generative AI APIs provide safety attribute scoring so teams can test safety filters and set confidence thresholds suited to their use case. Those controls operate on model input and output automatically. Human review adds judgment for cases those controls cannot resolve, such as specialized domain questions, edge cases, and decisions that could materially affect individual rights.
One area where automated protection is essential is rendered model output. If an application automatically renders model output such as Markdown or HTML in a browser or UI, it may be vulnerable to indirect prompt injection and data exfiltration. The recommended protections are to sanitize output with a strict HTML sanitizer such as DOMPurify before rendering, enforce Content Security Policy (CSP) headers to block external image loading and untrusted script execution, disable automatic rendering of external images or require user consent, and automatically append rel attributes to generated links.
Exam tip: Human supervision is specifically required in contexts with the potential to materially impact individual rights, and it appears as the last step of the recommended sequence — solicit user feedback and monitor content — not as a substitute for safety filters.
Prompt engineering refines inputs to guide a foundation model toward more accurate responses without changing model weights, while model fine-tuning further trains the base model on specialized data to adapt it for a specific task. Prompt engineering offers speed and efficiency with low compute, data, and cost, while fine-tuning requires more upfront investment but delivers more consistent specialized performance.
Model fine-tuning should be used instead of prompt engineering when the task is specialized, repeated at scale, and needs consistent accuracy or tone. Fine-tuning is justified when volume is steady, language is specialized, and prompt engineering cannot meet quality needs reliably after repeated testing.
Retrieval-augmented generation (RAG) reduces hallucinations by retrieving relevant external information and giving it to the model before generation so responses are based on verifiable sources. The ordered flow converts the user query into an embedding, compares it with document embeddings through similarity search, retrieves the most semantically relevant information, generates a response from that information, and then performs a grounding check before returning the response.
Human-in-the-loop (HITL) places a person inside the AI process to review, validate, or correct output before it is used. Human supervision is specifically required in contexts with the potential to materially impact individual rights and appears as the final step after assessing security risks, performing safety testing, and configuring safety filters, with feedback looping back to improve safeguards.
A customer service team wants to use a foundation model on Vertex AI to summarize incoming customer complaints and generate draft email responses. The team has no machine learning engineers, lacks a large labeled dataset, and needs to test and adjust instructions immediately with zero additional compute training costs.
Which approach should the team implement to achieve rapid results and refine outputs?
Last updated