Prepare and test your skills
Prepare and test your skills
Integration of search results with large language models (LLMs) is pre-built Retrieval-Augmented Generation (RAG) where search supplies fresh context for generation. In this design, Vertex AI Search (opens in a new tab) first retrieves relevant information from authoritative enterprise data, and then the LLM uses that retrieved context to write its answer. This connection lets the model answer from current company content instead of relying only on what it learned during training.
Grounding is the practice of tying a model's answer to specific, retrievable source content so the answer can be checked. Generative AI (opens in a new tab) models learn patterns from input training data and then generate new text, images, code, and other content with similar characteristics. A hallucination is a confident response that cannot be grounded in training data and may sound plausible while being factually incorrect. Pre-built RAG reduces this risk by placing verified enterprise passages alongside the user's question before the LLM generates a response.
Vertex AI Search is the retrieval side of pre-built RAG that finds relevant context for the LLM. Data flows from connected data stores, including unstructured data and website data, to the search service in response to the user query, and then selected extracts flow from the search response to the LLM as added context. The LLM then generates a new answer that is shaped by both the original question and the supplied passages. Systems that use a graph representation can further improve retrieval by traversing nodes and edges to retrieve relevant subgraphs, which gives the generator richer, relationship-aware context.
A process flow showing a user query flowing through Vertex AI Search, which retrieves verbatim extracts from data stores, then to an LLM that combines question and retrieved context to produce a grounded response with citations.
Snippets, extractive answers, and extractive segments are three kinds of verbatim extracts that Vertex AI Search can return with each search result. Snippets are short previews with hit highlighting in bold HTML tags, while extractive answers and extractive segments are longer verbatim sections taken directly from the source document. Extractive segments are usually more verbose than extractive answers and are designed to be displayed as answers, used for post-processing, and used as input for LLMs to generate answers or new text.
The request controls which output is returned. A search request sets returnSnippet to true for snippets, and sets maxExtractiveAnswerCount for extractive answers, where the default is 0 and the maximum is 5. The options compare as follows:
| Search output | What it provides | Typical use and availability |
|---|---|---|
| Snippet | Brief verbatim extract with bold hit highlighting, returned in derivedStructData.snippets with a snippet_status of SUCCESS or NO_SNIPPET_AVAILABLE | Preview text under each result to help users judge relevance; available for unstructured and website search |
| Extractive answer | Short verbatim paragraph, table, or list extracted from the document, returned in extractive_answers | Precise answer near the top of a page when verbatim text is preferable to a rephrased summary; requires Enterprise edition features, plus advanced website indexing for website data |
| Extractive segment | Longer verbatim passage than an extractive answer | Input context for an LLM and for post-processing tasks; requires Enterprise edition features, plus advanced website indexing for website data |
Extractive answers can serve as an alternative to summarized responses when exact wording matters, while extractive segments give the LLM broader context for synthesis. Snippets remain useful for result previews, but they are briefer and more uniform in length since July 2023 and no longer provide page numbers.
Exam tip: Use extractive answers, which provide page numbers where available, or extractive segments for LLM input instead of snippets when verifiable answers are needed.
A grounded response is an LLM answer that is constrained to the passages supplied by Vertex AI Search rather than invented from general training patterns. Because the model works from retrieved enterprise documents, its answer stays tied to current and authoritative facts, which minimizes hallucinations. The verbatim nature of the extracts also supports verification, since the response can point back to the source link, snippet, answer, or segment from which each fact came. In this way, retrieval provides the evidence and generation provides the readable summary, with citations that let a reader check the original document.
Strategic application of pre-built RAG means using a managed search service to ground generative AI in real-time enterprise data. Pre-built RAG with Vertex AI Search (opens in a new tab) is a fully managed and optimized way to add outside information to the model prompt, so responses are informed and accurate instead of relying only on training data. Vertex AI Search builds the search engine for RAG by indexing your data and serving relevant results to the model.
Vertex AI Search provides a search engine for RAG that indexes your data and supplies grounding passages to a generative model. The service supports most data sources, so teams can connect existing content without rebuilding storage or pipelines for each source type. The service also prevents direct access to the underlying embeddings, so applications query the search engine rather than reading raw vector data. This combination makes Vertex AI Search a pre-built path to RAG where retrieval, ranking, and embedding management are handled inside one managed service.
A managed RAG solution with Vertex AI Search reduces operational work because Google Cloud operates the optimized search and vector infrastructure. Teams do not need to design a tailor-made search engine, tune a specialized vector database, or manage embedding storage and serving separately. Access stays at the search-engine boundary because callers cannot directly reach the underlying embeddings, which simplifies how data is exposed to applications. Choose this managed path when the goal is grounded answers from enterprise data with less infrastructure to build and maintain.
Vertex AI Search fits internal and external knowledge discovery when answers must come from your own authoritative data. Internal uses include employee assistants that answer from company documents, policies, and knowledge bases with current information. External uses include customer-facing search and assistants that answer from product documentation, support content, and published data sources. In both cases, data flows from your indexed sources to Vertex AI Search and then into the model prompt, so the generative model responds with retrieved context.
Grounding data can also come from other paths, and the choice depends on where the trusted information lives:
Exam tip: Choose Vertex AI Search when you want a fully managed solution that supports most data sources and prevents direct access to the underlying embeddings.
Google Cloud grounding options differ by control and data pattern, and Vertex AI Search is the pre-built choice for most RAG search needs. The service depends on its managed index to serve results, while the other options depend on specialized databases or custom orchestration. The options compare as follows:
| Option | Management and control | Choose when |
|---|---|---|
| Vertex AI Search | Fully managed, optimized search engine | You want pre-built RAG that supports most data sources without exposing embeddings |
| Vertex AI RAG Engine | Managed orchestrator with LlamaIndex-like interface | You want to build a RAG search engine with orchestrator-level control |
| Vertex AI Vector Search | Specialized, optimized vector database | You need low-latency vector search and large-scale serving with a tailor-made design |
| AlloyDB and Cloud SQL | SQL database with direct model use | Data is accessed programmatically as OLTP and already lives in SQL with low-latency needs |
| BigQuery | Analytical SQL dataset | Data is a large OLAP dataset with batch processing and frequent SQL access |
Use Vertex AI Search first when the requirement is a managed search engine for RAG over diverse enterprise content, and move to the other options only when the pattern matches a specialized database or custom vector workload.
Data ingestion and indexing for retrieval is the process of bringing source data into a searchable knowledge base so a generative AI application can retrieve relevant content. In pre-built RAG with Vertex AI Search (opens in a new tab), Google Cloud provides a fully managed search engine that indexes your data for retrieval and prevents direct access to the underlying embeddings. The model then retrieves relevant information at generation time to ground its responses.
Vertex AI Search is a fully managed, optimized solution for grounding generative AI with your own data. The service supports most data sources and prevents direct access to the underlying embeddings, so applications query the search engine rather than managing the embedding layer themselves. The search engine uses an index of your data, so retrieval happens from current indexed content at generation time. This design makes Vertex AI Search a fit when you want a search engine for RAG without building or managing embeddings.
Indexing and embeddings for retrieval are how a search engine organizes data for similarity search. Grounding can be implemented by using an index of your data with a search engine, where many search engines store embeddings in a vector database. The vector database is an optimal format for similarity search because it lets retrieval find content that is semantically similar to a query. Vertex AI Search fits this managed pattern by providing the search and retrieval layer while keeping the underlying embeddings hidden from direct access.
Data sources and access boundaries define what content can be ingested and where users are allowed to interact with it. Vertex AI Search supports most data sources, which allows diverse enterprise data including unstructured and structured formats to be ingested into a searchable knowledge base. The trust boundary sits at the search and retrieval layer: users query the index, but they do not directly access the raw embeddings. If a team needs low-latency vector search, large-scale serving, or a specialized and optimized vector database, Vertex AI Vector Search is an option, and if they want a managed orchestrator with a LlamaIndex-like interface, Vertex AI RAG Engine (opens in a new tab) is an option.
Grounding without a vector database is possible when data already lives in an operational database. You can ground by querying an existing database such as Cloud SQL (opens in a new tab) or Firestore, then using the query result in the model prompt. This alternative is useful when data already lives in a relational or document database and does not need a vector index. For pre-built RAG with Vertex AI Search, however, the focus is on a managed search engine that indexes data for retrieval.
Exam tip: Vertex AI Search is the fully managed option that supports most data sources and prevents direct access to the underlying embeddings.
Pre-built Retrieval-Augmented Generation (RAG) with Vertex AI Search supplies fresh context for generation by first retrieving relevant information from authoritative enterprise data and then having the Large Language Models use that retrieved context to write the answer. Data flows from connected data stores to Vertex AI Search in response to the user query, and then selected extracts flow from the search response to the Large Language Models as added context. This lets the model answer from current company content instead of relying only on what it learned during training.
Snippets are brief verbatim extracts with bold hit highlighting used as preview text under each result to help users judge relevance, while extractive answers are short verbatim paragraphs, tables, or lists for precise answers and extractive segments are longer verbatim passages designed as input context for Large Language Models and for post-processing tasks. A search request controls the output with returnSnippet for snippets and maxExtractiveAnswerCount for extractive answers, where the default is 0 and the maximum is 5. Extractive answers and extractive segments require Enterprise edition features, plus advanced website indexing for website data.
Use Vertex AI Search when you want pre-built Retrieval-Augmented Generation (RAG) with a fully managed and optimized search engine that supports most data sources without exposing embeddings. Choose Vertex AI RAG Engine when you want to build a search engine with orchestrator-level control through a managed orchestrator with LlamaIndex-like interface, and choose Vertex AI Vector Search when you need low-latency vector search and large-scale serving with a tailor-made design. Choose AlloyDB and Cloud SQL when data already lives in SQL as OLTP with low-latency needs, and BigQuery when data is a large OLAP dataset with batch processing and frequent SQL access.
An enterprise is setting up pre-built Retrieval-Augmented Generation (RAG) using Vertex AI Search to ground a generative AI assistant in company knowledge. The organization needs to ingest:
When refreshing the search index, administrators must ensure that newly added documents are indexed, existing documents are updated, and obsolete records that were deleted from the source systems are automatically removed from the knowledge base.
Which ingestion approach should the team configure in Vertex AI Search to meet these requirements?
Store all unstructured PDFs inside structured BigQuery string columns before indexing
Configure an ad-hoc semantic embedding pipeline in BigQuery ML without creating a search data store
Import the structured and unstructured datasets into the data store using the FULL reconciliation mode
Import the datasets using the INCREMENTAL reconciliation mode during recurring refreshes
Last updated