Prepare and test your skills
Prepare and test your skills
Building secure AI systems is the layered defense that stops model weights, training data, and prompts from leaving authorized boundaries or being manipulated by attacker text. The defense combines network perimeters, private connectivity, least-privilege identity, encryption, audit logging, data de-identification, and prompt screening. Traffic flows from users and pipelines through enforcement points to VPC Service Controls (opens in a new tab) perimeters, identity checks, and model screening before it reaches Vertex AI (opens in a new tab) endpoints and data stores.
VPC Service Controls perimeters are service boundaries that enclose Vertex AI and data services inside a Virtual Private Cloud (VPC). The perimeter includes Vertex AI endpoints, datasets, pipelines, Model Registry, plus BigQuery and Cloud Storage, so API calls and data flows from inside the perimeter to an external project or the public internet are denied by default. Data flows from a training job or prediction endpoint to the outside only when an explicit ingress or egress rule allows it, which stops exfiltration even when an identity has valid credentials. Always validate a new perimeter in dry run mode before enforcement, because dry run logs what would be blocked without breaking legitimate pipeline or serving traffic.
Private connectivity is the network design that keeps model traffic off the public internet by moving it through private paths. Private Service Connect provides private endpoints that enable access to Vertex AI models from within a VPC network, and custom training jobs can use a Private Service Connect interface plus a private IP address to communicate privately with other authorized Google Cloud services. IAM is not applied to the private endpoint itself, but the target Vertex AI service uses IAM to manage access to the underlying model, so network reachability and identity authorization remain separate checks. Firewall rules add a second layer by denying all traffic by default and allowing only required ports, protocols, and sources.
To secure serving traffic at the application layer, front inference endpoints with Apigee for enterprise-grade governance or API Gateway for straightforward authentication. Apigee and API Gateway enforce API keys, authentication, authorization, and rate limiting and throttling, which controls request volume from a single IP address or user. Rate limiting matters because a sudden spike or thousands of slightly different sequential queries can indicate denial-of-service, model theft, or model inversion probing to reverse-engineer weights or recover training data.
Exam tip: A Vertex AI private endpoint controls network reachability inside the VPC, while IAM on the target Vertex AI service controls which identity can use the model — both must allow the request.
Fine-grained IAM roles are least-privilege grants that give each human, service account, and pipeline only the permissions it needs on specific endpoints, datasets, pipelines, and Model Garden models. Create dedicated service accounts for MLOps pipelines, for example a training service account that can read from only one Cloud Storage bucket and write only to Model Registry. Use IAM Conditions to enforce attribute-based access, such as allowing a pipeline trigger only if the request originates from a trusted Cloud Build trigger.
Additional controls narrow access further around sensitive AI data:
Encryption control with customer-managed encryption keys (CMEKs) gives the organization control over the keys that encrypt AI data at rest in Cloud Key Management Service (Cloud KMS). When CMEKs are used, CMEK-integrated services encrypt stored data for Vertex AI datasets, models, metadata stores, Feature Store, and endpoints, plus Cloud Storage, BigQuery, Cloud SQL, and Spanner. Key revocation or destruction then breaks decryption across artifacts, pipelines, and data stores, which halts further use of exfiltrated ciphertext. Encryption in transit protects prediction traffic between clients, load balancers, and AI APIs. Use HTTPS for all API calls and enforce HTTPS with HTTPS load balancers so features, prompts, and responses flow over encrypted channels.
Audit logging and detection is the observability layer that records Vertex AI actions and flags post-authentication anomalies. Cloud Audit Logs provides Admin Activity audit logs that are enabled by default and record endpoint creation, model deployment, and configuration updates. Data Access audit logs record every read and write, including who accessed data and models and when, but they must be explicitly enabled for Vertex AI and for stores such as BigQuery, Cloud Storage, Cloud SQL, AlloyDB, Firestore, Spanner, and Bigtable. Enable Data Access logging at the organization or project level, then analyze logs with Observability Analytics and route logs to long-term destinations for investigation.
Detection depends on baselining normal access and alerting on anomalies. Configure alerts in Cloud Logging for abnormal query patterns, sudden spikes in request volume from one user or IP, and geographic and temporal anomalies such as a massive spike in logins from a new country at 3 AM. Feed Model Monitoring (opens in a new tab) drift signals, endpoint traffic metrics, and audit findings back into retraining, access review, and perimeter updates, so a suspected probe triggers throttling, key rotation, or perimeter tightening before data leaves the boundary.
Data minimization is the practice of not collecting, keeping, or using data that is not strictly necessary for the business goal. For AI workloads, minimization means preferring synthetic or fully anonymized data for training and evaluation, and de-identifying any operational dataset that must retain utility. De-identification preserves analytical value while removing direct identifiers, so a model can learn patterns without learning identities.
Sensitive Data Protection (opens in a new tab) implements minimization through transformation methods:
Choose de-identification when training data, retrieval context, or logging pipelines must flow toward foundation models but must not carry raw PII or intellectual property.
Discovery and classification is the continuous process where Sensitive Data Protection inspects stored data, labels what it finds, and publishes results for governance. Inspection jobs scan sources such as BigQuery tables, Cloud Storage objects, and training datasets to discover PII and intellectual property. Findings then flow from Sensitive Data Protection into Knowledge Catalog as inspection-result aspects, where data lineage, data profiles, and business metadata make exposure visible to owners. Knowledge Catalog links a raw source to its processed datasets, feature metadata, and model artifacts, while BigQuery column-level lineage traces a sensitive column forward to the dataset version used for training.
Prompt and response screening is the inline enforcement where untrusted text is checked on entry to the model and rechecked on exit. Preprocessing screening validates user inputs, retrieved documents, and tool outputs before they reach the foundation model, blocking injection payloads and stripping PII. Postprocessing screening validates generated outputs before they reach the user, redacting secrets the model repeated or inferred. An expert deployment chains both checkpoints into one ordered flow:
DeidentifyTemplate for redaction.| Protection point | Direction of flow | Risk blocked |
|---|---|---|
| Preprocessing on prompts | User and retrieval data flows into Model Armor and then into the foundation model | PII, credentials, and intellectual property leaking into prompts, logs, or retrieval stores |
| Postprocessing on responses | Model output flows through de-identification and then to the user | Memorized PII or policy-violating content leaking out through generation |
Model Armor data protection is the platform-level control that enforces safety thresholds and mandatory de-identification on model endpoints. Model Armor screens prompts and responses for injection and harmful content, and its data_protection_config binds that screening to a Sensitive Data Protection DeidentifyTemplate. The template defines which detectors to run and whether to redact, mask, or tokenize each finding, so enforcement does not depend on application code. A configuration references the template by its full resource name in projects/PROJECT_NUMBER/locations/LOCATION/deidentifyTemplates/DLP_TEMPLATE_ID and pairs it with thresholds such as injection: HIGH and harmful_content: MEDIUM. Because the policy applies to the endpoint, sensitive information is redacted before return even if the model itself is compromised.
Lineage and access tracking is the governance layer that proves which sensitive data moved where and who touched it. Knowledge Catalog records lineage from raw source through transformation to dataset version and registered model version, while Model Registry links each model version to its training parameters, evaluation metrics, and source dataset. Cloud Audit Logs and Data Access audit logs capture API calls and data access events and flow into Cloud Logging for analysis and alerting. When an output incident occurs, teams trace backward from the prediction to the model version, to the dataset version, to the inspection finding that should have masked it.
Input validation in AI systems treats every prompt and every piece of retrieved data as untrusted until it is checked. Untrusted sources include end-user text and agent-fetched content from external tools, documents, and search results, which all flow from the outside into the model context. Secure coding validates and sanitizes both inputs and outputs in application code to prevent manipulation and system exploits. In ingestion scripts for agent pipelines or BigQuery, validation in Python checks data types, formats, and ranges before training or inference, so malformed or malicious values do not reach the model.
Screening with Model Armor and safety thresholds is the platform inspection of prompts before they reach the model and responses before they return. The request flow runs from the user prompt to Model Armor pre-screening to the Vertex AI endpoint to Model Armor post-screening to the user, with redaction applied even if the model is compromised. Model Armor is enabled on a serving endpoint with the Google Cloud CLI, which activates protection against injection and jailbreaking. Minimum thresholds and data protection are enforced through a configuration that sets safety_thresholds and points data_protection_config to a DeidentifyTemplate. As a complement, a fine-tuned large language model (LLM) can screen prompts and responses, with a human in the loop to validate changes proposed by coding assistants.
Secure agent design limits model freedom because open-ended plan and execute architectures let injected text steer control flow. The action-selector pattern restricts the model to translating a request into one of a small, pre-defined set of safe functions, while hard-coded logic outside the LLM cannot be modified by prompt text. The dual-LLM pattern separates duties between a primary action LLM and a secondary guardrail LLM that pre-screens prompts and post-screens outputs.
| Pattern | Model role | How injection is contained |
|---|---|---|
| Action-selector pattern | Model selects one safe function | Control flow lives in hard-coded code, so injected text cannot create new actions |
| Dual-LLM pattern | Action LLM acts, guardrail LLM checks | Guardrail LLM blocks malicious prompts before execution and leaks after execution |
Choose the action-selector pattern when the task maps to a closed set of operations, and add the dual-LLM pattern when free-form reasoning is required but every input and output must still be checked.
Tool governance ensures an agent calls only the tools necessary for its task and cannot expand its own privileges. Orchestration code must prohibit dynamic tools, meaning the agent cannot register new tools or change permissions. Allowlist enforcement declares permitted functions or tables in both the system prompt and backend code, so the backend denies anything outside the list even if the prompt is manipulated. General database tools create the highest leakage risk in multi-tenant applications. A general execute_sql tool lets the caller query any data that permissions allow, so a malicious user can trick the agent into reading another user's rows. The safe alternative builds narrow custom tools with MCP Toolbox for Databases (opens in a new tab), where the identity filter is set outside agent control, for example a lookup_active_order tool where backend code injects the authenticated user ID into the query filter.
Post-authentication defense assumes an attacker has bypassed authentication and then attacks internally through network, application, and inference layers. VPC Service Controls creates a perimeter around Model Registry, BigQuery, and Vertex AI resources to prevent exfiltration, with ingress and egress rules controlling flow and dry run mode validating impact. Identity-Aware Proxy protects web tools such as notebooks, while Apigee provides request validation and traffic control for inference endpoints, with API Gateway as the lighter option. Behavioral monitoring detects probing that passes initial checks:
Auditing and evaluation provide visibility into what the agent did and whether injected inputs could succeed. Cloud Audit Logs must have Data Access audit logs enabled for Model Context Protocol (MCP) and for BigQuery, Cloud SQL, AlloyDB, Firestore, Spanner, and Oracle Database@Google Cloud, because only Admin Activity logs are enabled by default. Data Access logs record every read and write by the agent service account, and Cloud Logging alerts can trigger on high-risk patterns such as service-account writes to Firestore. Resilience is tested by checking for data corruptions, running adversarial attack simulations, verifying container signatures with Binary Authorization, and using Model Monitoring to detect drift and skew that may signal tampering.
Aligning with responsible AI practices is the lifecycle discipline of measuring fairness, explaining predictions, and monitoring production for emerging bias. A model can show strong aggregate accuracy while performing poorly for a specific slice, so overall metrics must always be compared with disaggregated slice metrics. In Google Cloud, this work connects data validation, Vertex AI Model Evaluation (opens in a new tab) in pipelines, explainability, and continuous monitoring with governance in Model Registry.
Data bias and model bias indicators are evaluation signals that separate skew in the dataset from disparate behavior in the trained model. Gemini Enterprise Agent Platform exposes metrics called data bias and model bias that help evaluate fairness across categories such as race, gender, and class. Data bias reflects skew present in the dataset before training, while model bias reflects disparate behavior in predictions after training. Statistical deviations across categories do not automatically prove harm, because differences might reflect sample size, valid base rates, or domain context, so deviations require domain review rather than automatic rejection.
Sliced evaluation is the comparison of overall model performance against performance computed on predefined slices defined by demographic groups or sensitive attributes. Sliced evaluation starts by defining fairness measures that match the use case, such as demographic parity, equalized odds, or disparate impact, then evaluating the model for biases across different slices. This disaggregated view identifies disparate performance that affects specific subgroups even when aggregate metrics look acceptable. Proactive work spans the MLOps lifecycle:
The result for a slice depends on the slice definition, metric, and dataset, so the same model version must be compared on consistent slices over time.
Vertex AI Model Evaluation is the Agent Platform capability for scoring model versions against defined metrics and datasets and tracking results for lineage. The evaluation can run as a pipeline component with Agent Platform Pipelines, either by itself or with other components for data processing, training, evaluation, and deployment. The evaluation component consumes a model version and an evaluation dataset, computes overall and sliced metrics, and the workflow then compares model versions. Evaluation results flow from the evaluation component to Agent Platform ML Metadata for lineage, which links metrics back to training parameters, dataset version, and code version and feeds re-evaluation or retraining.
Explainability is the set of techniques that show which inputs drove a prediction so teams can check whether drivers align with domain knowledge. Vertex Explainable AI (opens in a new tab) provides feature-based explanations and is integrated with both BigQuery ML and Agent Platform. For tabular models on Agent Platform endpoints, feature attributions indicate the input features that contributed most to each prediction, which helps reveal when a sensitive or proxy feature carries unexpected weight. Additional diagnostic practices connect explanations to decisions:
Explanations do not by themselves prove bias, but they show where to investigate when a slice underperforms.
Continuous fairness monitoring is the production practice of detecting shifts in behavior or outputs that might indicate emerging bias after deployment. Model Monitoring on Agent Platform (opens in a new tab) tracks prediction drift, data drift relative to training, shifts in prediction distributions, and spikes in low-confidence predictions, then feeds insights back into re-evaluation or retraining. Cloud Logging provides an audit trail of key events, while Error Reporting groups errors to reveal patterns that point to bias. Governance preserves evidence, because data lineage from Knowledge Catalog (opens in a new tab) tracks where data comes from and what transformations are applied, while Model Registry stores each version with its parameters, metrics, dataset version, and limitations.
Sampled Shapley attribution is the Vertex Explainable AI method for structured data that approximates exact Shapley values from cooperative game theory. Shapley values treat each feature as a player in a game and assign credit in proportion to the outcome of an inference. Exact computation considers different permutations of features, which is intractable for large feature sets, so the service uses a sampling approximation called Sampled Shapley. Increasing the number of sampled paths increases precision, but attributions can change dramatically as precision changes, so low path counts produce unstable explanations.
The explanationSpec is the model configuration that controls how Vertex Explainable AI computes attributions for online and batch predictions. The explanationSpec.parameters block sets the method and precision, for example sampledShapleyAttribution.pathCount: 6 controls how many permutation paths are sampled and topK: -1 controls how many highest-attribution features are returned. The explanationSpec.metadata block defines the contract, with metadata.inputs mapping each input name such as prediction_values to its inputBaselines, metadata.outputs naming the predicted output such as value, and featureAttributionsSchemaUri describing the output schema. Data flows from the baseline input through the model to the inference output, and attribution is the change in inference value relative to the baseline, so the baseline choice determines the question the explanation answers.
BigQuery ML (BQML) explainability separates local per-row explanations from global dataset-level explanations using extended SQL functions. Local functions return attributions for each row, while global functions aggregate attributions across the evaluation dataset to support bias review. ML.GLOBAL_EXPLAIN returns global explainability by taking the mean absolute attribution each feature receives for all rows, which surfaces features the model systematically relies on.
| Model type | Attribution method | Local function | Global function |
|---|---|---|---|
| Boosted tree and random forest | Tree SHAP and Gini importance | ML.EXPLAIN_PREDICT | ML.FEATURE_IMPORTANCE and ML.GLOBAL_EXPLAIN |
| AutoML Tables | Sampled Shapley | Not applicable as separate local function | ML.GLOBAL_EXPLAIN |
| ARIMA_PLUS time series | Decomposition into trend, seasonal, holiday, and changes | ML.EXPLAIN_FORECAST | Not applicable |
| ARIMA_PLUS_XREG multivariate | Decomposition plus Shapley for regressors | ML.EXPLAIN_FORECAST | Not applicable |
The table shows that tree models support both local and global views, AutoML Tables explanations are aggregated through ML.GLOBAL_EXPLAIN, and forecasting models explain through decomposition. Choose a global function when asking whether a proxy consistently drives outcomes, and a local function when asking why a single prediction changed.
Attribution interpretation for bias review starts from the rule that attributions only express how much a feature moved the prediction away from the baseline prediction. A large positive or negative attribution means that feature moved the prediction from the baseline, not that the feature is inherently important in isolation. When a sensitive attribute or proxy repeatedly receives high absolute attribution in global results, the model depends on that encoding of historical bias. Counterfactual sensitivity is tested by changing the baseline or feature value and observing whether attribution and prediction shift, which reveals whether the outcome is fragile to that feature.
Data skew and data drift are two distribution shifts that signal performance and fairness risk in production systems. Data skew occurs when the distribution of feature values for training data is significantly different from serving data in production. Data drift occurs when feature distribution in production changes significantly over time. A model performs best on serving data similar to training data, so understanding variance between training and serving data is critical for accuracy and equity.
| Shift | Comparison source to destination | When it appears |
|---|---|---|
| Data skew | Training statistics flow to serving statistics for comparison | At deployment or when production inputs differ from training |
| Data drift | Earlier serving statistics flow to later serving statistics | Over time in production, for example between days |
Exam tip: Data skew compares training data to serving data, while data drift compares one span of serving data to another span of serving data.
Monitoring functions in BigQuery ML are SQL functions that compute descriptive statistics and validate differences without separate monitoring infrastructure. Training statistics are saved during model training, so the original training data is not required for skew detection. Statistics for skew checks are only computed for serving feature columns that match training feature columns, which improves performance and lowers cost. The functions divide by purpose:
tfdv.generate_statistics_from_csv for integration.validate_statistics for training-to-serving or serving-to-serving checks.The skew monitoring workflow is the ordered routine for detecting when production inputs have diverged from training inputs. The workflow depends on training statistics saved by BigQuery ML at model creation, which become the baseline that serving statistics flow into for comparison. The workflow runs first through validation, then remediation:
ML.DESCRIBE_DATA on training and serving data to confirm expected parameters.ML.VALIDATE_DATA_SKEW to compare serving statistics against saved training statistics.Only after the root cause is addressed does retraining restore alignment between model and serving environment.
The drift monitoring loop is the continuous routine that watches production data changing over time before datasets diverge too much to retrain. Drift detection in BigQuery ML compares consecutive spans of serving data, for example between different days, and lets teams receive notification while correction is still practical. The loop starts with ML.DESCRIBE_DATA to confirm datasets remain within expected parameters. When ML.VALIDATE_DATA_DRIFT or ML.TFDV_VALIDATE flags a difference between serving windows, data flows from the alert into investigation of shifted features, then into adjustment of training data and retraining. The control boundary sits between serving and training: serving statistics are observed continuously, while saved training statistics remain the fixed reference until a new model version replaces them.
Continue with this study guide: Aligning with responsible AI practices (e.g., monitoring for bias)
Model explainability on Agent Platform is the ability to reconstruct why an agent or model produced an output and to monitor whether those explanations remain trustworthy in production. Tabular predictions are explained by feature attributions, while agent decisions are explained by the ordered process of calls and reasoning steps. On Gemini Enterprise Agent Platform (opens in a new tab), this combines trace data, evaluation, and monitoring that feeds back into retraining or rollback.
Agent execution decomposition breaks one agent response into linked model calls, tool invocations, and reasoning transitions. Data flows from the user request to the orchestrating model, then to each selected tool or retrieval system, then back to the model for synthesis, and finally to the user-facing output. Each handoff is a decision point where the agent selects the next model, tool with its arguments, or reasoning branch. When output is unsafe, biased, or factually wrong, decomposition shows whether failure came from model reasoning, a tool return, retrieved context, or prompt construction. For RAG agents, retrieval needs separate checks for relevance of retrieved context and groundedness of the response, supported by the Gen AI evaluation service for accuracy, safety, toxicity, and format adherence.
Trace logs are the primary explainability mechanism for complex LLMs and agents because they record the process the agent followed. A complete trace names the source model call, each destination tool or subordinate model, the inputs and outputs at each boundary, and the order of execution. Prompt engineering can guide the model to expose reasoning steps, while RAG applications should return citations that link each claim to its source. Production capture depends on Cloud Logging, Cloud Trace, and Cloud Monitoring working together:
Audit trails record who accessed models or tools, what data was used for prompting, and what queries reached endpoints. Vertex ML Metadata and Model Registry preserve lineage from raw data to datasets to model versions, letting an operator move backward from a bad step to the exact artifact that produced it.
Vertex AI Model Monitoring (opens in a new tab) is the managed service for detecting training-serving skew, data drift, and prediction drift for deployed models. The service compares live inference data against the training baseline and sends alerts when incoming data skews too far, which signals that retraining may be needed. For agent endpoints, monitoring also tracks shifts in prediction distributions and spikes in low-confidence predictions that often precede degraded reasoning. Continuous output monitoring applies the same idea to generative behavior, watching for unsafe, biased, or off-topic content, with Model Armor screening prompts and responses and granular validation checking outputs against expected ranges and schemas.
Exam tip: Use feature attributions for tabular models deployed to Agent Platform endpoints, and use trace logs for LLM agents where explainability means reconstructing the process the agent followed.
Fault isolation is the ordered workflow that links a production symptom to one failing step in the agent chain. The workflow depends on the trace, monitoring alerts, and lineage records all referencing the same request and model version. Reconstruction comes first, then testing of each component in isolation:
Remediation such as prompt revision, tool replacement, retrieval tuning, or retraining happens only after the source step is identified. For tabular sub-models, Vertex Explainable AI adds feature-level evidence with integrated gradients for neural networks and sampled Shapley for tree models.
Explanation methods divide by model type because tabular predictions and agent decisions fail in different ways. Tabular explainability attributes one prediction to its input features, while agent explainability attributes one outcome to a chain of calls and decisions.
| Option | Primary unit | Choose when |
|---|---|---|
| Vertex Explainable AI for tabular models | Feature attribution for one prediction | The question is which input features drove a classification or regression |
| Trace logs for LLM agents | Ordered process of calls, tools, and steps | The question is which step caused error, bias, or unexpected output |
Both forms belong in monitoring dashboards and, where trust requires it, exposed through application interfaces. Continuous monitoring then detects shifts that indicate emerging bias, completing the loop from observation back to troubleshooting.
Explanation signals for Agent Platform endpoints are the outputs that show why a model acted as it did. For tabular models, feature attributions indicate which inputs contributed most to each prediction. For generative models, explainability means explaining the process with trace logs, retrieved context, and citations in RAG applications. The What-If Tool, which integrates with TensorBoard, lets teams explore behavior without changing production code. Explanation quality degrades in three ways: stability means similar inputs produce similar attributions over time, consistency means the same version produces the same reasoning across slices, and fidelity means the explanation still matches what the model used. Documentation tracks shifts in top-ranked features, prediction distributions, low-confidence spikes, toxicity, grounding failures, and disparate performance across slices.
Drift detection on Model Monitoring compares live production data against the training baseline to catch changes that undermine explanations. A drift job runs on a schedule against logged predictions and features, evaluates distance metrics, and emits results to Cloud Monitoring for dashboards and alerts. The job depends on versioned training data and prediction logs, so Model Registry lineage linking each version to its dataset, parameters, and metrics is required to interpret any shift.
| Drift type | What is compared | What it signals for explanations |
|---|---|---|
| Data drift | Live input distribution compared to training distribution | Attributions may shift because the model sees unfamiliar values |
| Prediction drift | Live output distribution compared to training or prior serving | Decision boundaries moved, so prior explanations may no longer hold |
| Explanation drift | Live attribution or trace patterns compared to validated baseline | Top features or reasoning steps changed, indicating loss of stability or fidelity |
Data drift is detected by validating inputs against expected ranges and tracking distributions with TensorFlow Data Validation (TFDV), BigQuery, and Dataflow. Prediction drift is detected by monitoring output distributions, output schema, toxicity, and coherence. Explanation drift is inferred when those signals combine with reordered attributions, new trace paths, failed grounding checks, and fairness measures that diverge across slices.
Automated checks turn explanation monitoring into metrics, dashboards, and notifications that trigger action. First, the team configures drift detection jobs with thresholds for data drift, prediction drift, latency, throughput, and error rates. Then results flow from Model Monitoring into Cloud Monitoring, where they join business KPIs and log-based metrics from Cloud Logging. Finally, alert policies route violations to notification channels such as email, Slack, or PagerDuty and to automated remediation such as scaling, rollback, or retraining. Operational checks that support trust include:
Validation results are stored in BigQuery for trend analysis, while detailed logs are kept in Cloud Logging to debug specific requests. For generative endpoints, real-time events flow from the endpoint check to Pub/Sub, which fans out to alerting and incident management.
Monitoring output feeds back into continuous training and governance rather than ending at an alert. When data patterns change or fidelity drops, the alert triggers an Agent Platform Pipelines workflow that revalidates data, re-evaluates the model, and retrains on updated data. Evaluation results are logged to ML Metadata for lineage and to Model Registry as versioned metadata, including parameters, dataset version, and documented limitations and biases. Controlled deployment protects trust during transition. Traffic splitting supports rolling, canary, and A/B testing, so a small subset of traffic first flows from the endpoint to the new version while performance is monitored. Automated rollback reverts traffic to the previous stable version if alerts fire, while pipeline gates re-run explainability and responsible AI checks before any shift.
The AI-assisted development workflow connects a local IDE, the Looker extension, a managed context server, and prebuilt instructions. Your IDE and AI Agent is a local editor such as VS Code, Cursor, or Claude Code with an integrated agent. The Looker extension for VS Code provides syntax highlighting, autocomplete, bidirectional file synchronization, and integrated validation. The Looker-managed MCP server uses Model Context Protocol (MCP) to connect agents directly to the enterprise database and LookML models. Prebuilt skill files give the agent coding standards and project instructions. Data flows from the database and LookML models through the MCP server to the agent in the IDE, which reads local files, inspects schema, applies changes locally, and runs validation before commit.
Prerequisites establish authentication, context access, project content, and permissions before prompting begins. The setup depends on all four elements being complete, because the agent cannot inspect schemas or edit models without them.
develop permission for the model to be edited.Only after these steps does natural language prompting produce usable LookML, because the agent needs local files, live schema, and write permission together.
Prompting the agent uses natural language plus workspace skills to generate, refactor, and troubleshoot LookML. Clear prompts that name the connection, tables, files, and desired outputs yield the best results. The agent works from local files and live schema context rather than static examples.
ecommerce_db, inspect users and orders, generate view files with primary keys and measures, then generate a model that explores orders and joins users on user_id.products.view.lkml, find number dimensions for price, and generate matching sum and average measures.Inaccessible view: users, and asking the agent to review model and view files and propose a fix.In each case the level of detail in the prompt controls the quality of the generated LookML.
Review, validation, and deployment move agent-proposed edits from the local workspace to the development branch and then to production. The local workspace is the source, the development branch on the Looker server is the sync target, and Git is the deployment path.
Auto-labeling with a Document AI custom processor uses defined field names and prior annotations to label documents at scale. The processor depends on descriptive field definitions, imported documents, and human review to produce training-ready data. Field definition starts on the Get Started tab with Create new field, where a descriptive name and property description provide context and text patterns for each entity. Document import happens on the Build tab with Import documents, where the document path, target set, auto-labeling option, and foundation model are selected. Review happens under Build and Manage Dataset, where predictions are highlighted in purple for correction and missing fields are added. After review, select Mark as labeled and ensure the document is in either the Training or Testing set, because inaccurate fields will affect model performance.
data_protection_config and a DeidentifyTemplate enforces tokenization, masking, or redaction on prompts before the model and on responses after the model.execute_sql tool allows injection-driven cross-tenant reads, while a narrow custom tool enforces tenant isolation by injecting the authenticated user ID in backend code outside agent control.Data skew occurs when the distribution of feature values for training data is significantly different from serving data in production, while data drift occurs when feature distribution in production changes significantly over time. In BigQuery ML monitoring, ML.VALIDATE_DATA_SKEW compares serving statistics to saved training statistics, while ML.VALIDATE_DATA_DRIFT compares earlier serving statistics to later serving statistics.
Virtual Private Cloud Service Controls perimeters enclose Vertex AI endpoints, datasets, pipelines, Model Registry plus BigQuery and Cloud Storage so API calls and data flows to an external project or the public internet are denied by default. Data flows to the outside only when an explicit ingress or egress rule allows it, which stops exfiltration even when an identity has valid credentials.
Use Vertex Explainable AI for tabular models when the question is which input features drove a classification or regression, because feature attributions indicate which inputs contributed most to each prediction. Use trace logs for large language model agents when the question is which step caused error, bias, or unexpected output, because traces record the ordered process of model calls, tool uses, and reasoning steps.
Model Armor screens prompts before they reach the foundation model and responses before they return to the user, and its data protection config binds that screening to a Sensitive Data Protection DeidentifyTemplate. The template defines which detectors to run and whether to redact, mask, or tokenize each finding, so preprocessing blocks PII, credentials, and intellectual property from entering prompts and postprocessing redacts memorized PII or policy-violating content before return even if the model itself is compromised.
Last updated