A data engineering team is implementing a Retrieval-Augmented Generation (RAG) system on Google Cloud. They use Vertex AI text embedding models to generate high-dimensional vector embeddings of unstructured technical articles and index them using Vertex AI Vector Search with the Tree-AH algorithm. During evaluation, the team discovers that while query response latency is extremely low, the index's Approximate Nearest Neighbor (ANN) search recall is insufficient, causing relevant context to be missed during generative AI grounding. Which configuration change should the team make to improve search recall?

Increase the leaf_nodes_to_search_percent parameter in the Tree-AH algorithm configuration: What the Solution Is Vertex AI Vector Search utilizes the Tree-AH (Tree Asymmetric Hashing) algorithm based on Google's ScaNN library to provide high-speed, scalable Approximate Nearest Neighbor (ANN) search. The leaf_nodes_to_search_percent parameter specifies the percentage of index leaf nodes evaluated during a nearest neighbor query. How It Meets the Requirement Improve Search Recall: By default, Vector Search evaluates a subset (typically around 10%) of the index's leaf clusters to minimize query latency. Increasing leaf_nodes_to_search_percent instructs the search engine to traverse and scan a larger proportion of candidate leaf nodes, directly expanding the candidate search pool. Enhance Retrieval Accuracy: A higher percentage significantly reduces the probability of missing semantically relevant nearest neighbors that lie close to cluster boundaries, thereby boosting recall for Retrieval-Augmented Generation (RAG) grounding workflows. Tunable Trade-off: Increasing this value provides a direct, tunable dial between search recall and query latency without requiring a complete re-indexing or re-embedding pipeline. Key Advantages and Capabilities Direct Recall Control: Allows granular tuning of retrieval precision by scaling the percentage from 1 to 100. Hardware Acceleration: Tree-AH uses hardware-optimized asymmetric hashing to scan expanded leaf node partitions with minimal latency overhead. Preserves Data Architecture: Modifying the search exploration parameters does not alter the underlying dimension or structure of the generated text embeddings. Optimized for RAG: Ensures that the downstream Large Language Model (LLM) receives the most relevant retrieved context to mitigate hallucinations. Why This Is Optimal Adjusting leaf_nodes_to_search_percent directly targets the trade-off between recall and latency in Tree-AH indexes. It provides the necessary retrieval precision for generative AI pipelines while retaining the scalability benefits of approximate nearest neighbor indexing.

Tune Vertex Ai Vector Search Recall | Professional Cloud Data Engineer

AdvancedProfessional Cloud Data Engineer

A financial enterprise is designing a scalable ingestion pipeline on Google Cloud to prepare multi-page PDF prospectuses and scanned image documents for a Retrieval-Augmented Generation (RAG) system. The documents are stored in Cloud Storage and contain multi-column text, nested tabular structures, and complex visual formatting that must be preserved to perform accurate layout-aware chunking before generating vector embeddings. Which ingestion and extraction architecture should the data engineering team implement?

Preparing unstructured data for embeddings and retrieval-augmented generation (RAG)

AdvancedProfessional Cloud Data Engineer

A data engineering team is designing an unstructured data ingestion pipeline on Google Cloud for an enterprise Retrieval-Augmented Generation (RAG) application. The source corpus contains lengthy technical manuals and complex policy documents that feature hierarchical section headers, tables, and proprietary part numbers. During initial testing, the team observes two major retrieval issues: Queries targeting information split across section boundaries suffer from semantic context fragmentation and lost continuity. Queries containing exact proprietary part numbers and requests filtered by department or document version yield poor retrieval relevance. Which strategy should the team implement in their ingestion and retrieval pipeline to resolve both issues?

Preparing unstructured data for embeddings and retrieval-augmented generation (RAG)

AdvancedProfessional Cloud Data Engineer

A data engineering team is designing an automated CI/CD pipeline in Google Cloud to build and deploy custom containerized data processing applications. The solution must adhere to enterprise security and deployment standards: Container images must be stored immutably and uniquely identifiable in Artifact Registry to ensure reproducible deployments. The build service account used in the CI/CD pipeline must adhere strictly to the principle of least privilege. Sensitive third-party API keys needed by the workload must be securely retrieved without hardcoding or exposing them in plaintext within build configuration files or container image layers. Which combination of actions should the team implement?

CI/CD (Continuous Integration and Continuous Deployment)

AdvancedProfessional Cloud Data Engineer

An enterprise maintains a central data lakehouse architecture managed by Dataplex Universal Catalog, consisting of native BigQuery analytical tables and raw semi-structured data residing in Cloud Storage bucket assets. The compliance team requires an automated solution to: Continuously discover, profile, and classify sensitive personally identifiable information (PII) across both BigQuery and Cloud Storage. Enforce fine-grained column-level access controls and dynamic data masking so that non-privileged query users see masked values without altering the underlying data. Standardize policy management across all storage layers without maintaining disparate security pipelines. Which configuration strategy should the data engineering team implement to meet these governance requirements?

Data staging, cataloging, profiling, and discovery (data governance)

AdvancedProfessional Cloud Data Engineer

An enterprise is designing a secure data ingestion and analytics pipeline on Google Cloud that must adhere to strict regulatory compliance and data sovereignty constraints: Ingestion Staging (Cloud Storage): Compliance mandates that raw file uploads use encryption where cryptographic key material is held exclusively by the customer, supplied in-memory on each API request, and never stored persistently anywhere on Google Cloud servers. Analytics Warehouse (BigQuery): The data warehouse must use dedicated symmetric keys managed within Google Cloud with customer control over rotation schedules, audit logging of key access, and the capability to perform crypto-shredding on demand. Data Sovereignty Constraint (Regulatory HYOK): Master key material for specific regulatory analytics tables must physically reside within an external on-premises key manager outside of Google Cloud, while still integrating with Google Cloud services for server-side decryption. Which combination of encryption mechanisms should the enterprise implement?

Data encryption

FundamentalAssociate Data Practitioner

Storage Transfer Service Synchronization Options

Load data into Google Cloud storage systems using the appropriate tool (e.g., gcloud and BQ CLI, Storage Transfer Service, BigQuery Data Transfer Service, client libraries)

Professional Cloud Data Engineer

Tune Vertex Ai Vector Search Recall

Related questions

Storage Transfer Service Synchronization Options

Professional Cloud Data Engineer

Tune Vertex Ai Vector Search Recall

What the Solution Is

How It Meets the Requirement

Key Advantages and Capabilities

Why This Is Optimal

Related questions

Storage Transfer Service Synchronization Options

What the Solution Is

How It Meets the Requirement

Key Advantages and Capabilities

Why This Is Optimal