Monitoring AI Models, Drift and Safety

Identifying risks to AI solutions is the layered defense that keeps model weights, training data, prompts, and outputs inside authorized boundaries and prevents attacker text from steering behavior. Traffic flows from users and pipelines through network enforcement, identity checks, and content screening before it reaches retrieval-augmented generation (RAG) (opens in a new tab) systems, Vertex AI endpoints, and data stores. Each layer blocks a different escape path, so a failure in one check is still caught by another.

1.1Network Perimeters and Private Serving Paths

Network perimeters and private serving paths keep AI traffic off the public internet and block exfiltration even when credentials are valid. VPC Service Controls encloses Vertex AI endpoints, datasets, pipelines, Model Registry, BigQuery, and Cloud Storage inside a Virtual Private Cloud (VPC) boundary, so API calls and data flows from inside the perimeter to an external project or the internet are denied by default and allowed only by explicit ingress or egress rules. New perimeters should first run in dry run mode, which logs what would be blocked without breaking legitimate pipeline or serving traffic. Private connectivity moves the remaining allowed traffic through private paths, where Private Service Connect provides private endpoints to Vertex AI models from inside the VPC and training jobs use a Private Service Connect interface with a private IP to reach other Google Cloud services.

Firewall rules deny all traffic by default and allow only required ports, protocols, and sources, while serving endpoints are fronted by Apigee for enterprise governance or API Gateway for straightforward authentication on Cloud Run (opens in a new tab). Apigee and API Gateway enforce API keys, authentication, authorization, and rate limiting and throttling, which controls request volume from one IP or user. Rate limiting matters because sudden spikes or thousands of slightly different sequential queries can signal denial-of-service, model theft, or model inversion probing to recover weights or training data.

1.2Identity Boundaries, Encryption, and Audit Detection

Identity boundaries, encryption, and audit detection control who can reach AI resources, what they can decrypt, and whether anomalies are recorded. Fine-grained Identity and Access Management (IAM) gives each human, service account, and pipeline only the permissions it needs on specific endpoints, datasets, pipelines, and Model Garden models, such as a training service account that reads one Cloud Storage bucket and writes only to Model Registry. Access is narrowed further with IAM Conditions for attribute-based access, row-level and column-level controls in BigQuery, principal access boundary policies that limit reachable resources, Privileged Access Manager for just-in-time elevation, Identity-Aware Proxy policies based on IP and device for notebooks, and Vertex AI predefined or custom roles.

Customer-managed encryption keys (CMEKs) in Cloud Key Management Service (Cloud KMS) encrypt AI data at rest for Vertex AI datasets, models, metadata, Feature Store, endpoints, Cloud Storage, BigQuery, Cloud SQL, and Spanner, so key revocation breaks decryption across artifacts and pipelines. Encryption in transit protects features, prompts, and responses between clients, load balancers, and AI APIs through HTTPS and HTTPS load balancers. Cloud Audit Logs provides Admin Activity logs enabled by default for endpoint creation and deployment, while Data Access logs that record every read and write must be explicitly enabled for Vertex AI, BigQuery, Cloud Storage, Cloud SQL, AlloyDB, Firestore, Spanner, and Bigtable, then analyzed with Observability Analytics and alerts for volume spikes and geographic or temporal anomalies.

1.3Data Minimization, Discovery, and Lineage

Data minimization, discovery, and lineage reduce how much sensitive data reaches models and prove where it flowed. Minimization means preferring synthetic or fully anonymized data and de-identifying operational data that must retain utility. Sensitive Data Protection implements this with pseudonymization that replaces identifiers with reversible tokens, de-identification that removes or masks PII while keeping format or statistics, and generalization such as replacing exact ages with age ranges.

Discovery continuously scans BigQuery tables, Cloud Storage objects, and training datasets for PII and intellectual property, then publishes findings to Knowledge Catalog as inspection-result aspects with lineage, profiles, and business metadata. BigQuery column-level lineage traces a sensitive column forward to the dataset version used for training, while Knowledge Catalog links raw sources to processed datasets, features, and Model Registry versions. On Google Kubernetes Engine (GKE) (opens in a new tab), the same lineage lets teams trace backward from a bad prediction to the model version, dataset version, and inspection finding that should have masked it.

1.4Prompt Screening and Model Armor Enforcement

Prompt screening and Model Armor enforcement check untrusted text on entry to the model and recheck generated text on exit. Preprocessing validates user inputs, retrieved documents, and tool outputs before the foundation model, while postprocessing validates outputs before the user, so memorized PII or policy-violating content is redacted even if the model is compromised. The ordered flow is:

1. Inspect on ingress: Data flows from the user prompt and retrieval context to Sensitive Data Protection inspection and Model Armor screening.
2. Transform on ingress: Data flows from the screening decision to de-identification, where regex detectors and infoTypes trigger tokenization, masking, or redaction.
3. Invoke the model: Data flows from the de-identified prompt to the foundation model, so the model only receives the safe form.
4. Transform on egress: Data flows from the raw response back through Model Armor and the DeidentifyTemplate for redaction.
5. Deliver and log: Data flows from the de-identified response to the user, while access logs flow to Cloud Logging for audit.

Model Armor binds screening to a Sensitive Data Protection DeidentifyTemplate through data_protection_config, which names the template resource and sets thresholds such as injection: HIGH and harmful_content: MEDIUM. Because the policy applies to the endpoint, enforcement does not depend on application code.

1.5Secure Agent Autonomy and Tool Governance

Secure agent autonomy and tool governance limit what an agent can do when injected text tries to steer it. Open-ended plan-and-execute designs are risky because retrieved content flows from external tools, documents, and search results into model context as untrusted input. The action-selector pattern restricts the model to choosing one of a small set of safe functions while hard-coded logic outside the large language model (LLM) controls execution, and the dual-LLM pattern pairs an action LLM with a guardrail LLM that pre-screens prompts and post-screens outputs. Tool code must prohibit dynamic tool registration, enforce an allowlist of permitted functions or tables in both system prompt and backend code, and avoid general execute_sql tools that can read another tenant's rows. The safe alternative builds narrow custom tools with MCP Toolbox for Databases, where backend code outside agent control injects the authenticated user ID, for example a lookup_active_order tool filtered to the caller.

Post-authentication defense assumes an attacker passed login and probes internally, so VPC perimeters, Identity-Aware Proxy, Apigee traffic controls, and behavioral monitoring work together. Monitoring flags thousands of similar sequential queries, spikes from one IP or user, unusual geography or time, and AI inventory risk found by AI Protection. Resilience is tested with adversarial simulations, drift signals, container signature checks with Binary Authorization, and Data Access logging for computer use (opens in a new tab) and database tools.

1.6Fairness Evaluation and Explainability for Bias Review

Fairness evaluation and explainability for bias review separate dataset skew from disparate model behavior and show which inputs drove outcomes. Sliced evaluation compares overall performance to performance on slices defined by demographic groups or sensitive attributes using measures such as demographic parity, equalized odds, or disparate impact. Gemini Enterprise Agent Platform exposes data bias for skew before training and model bias for disparate predictions after training, but statistical deviations still require domain review for sample size and base rates. Vertex AI Model Evaluation runs as a pipeline component that consumes a model version and evaluation dataset, computes overall and sliced metrics, and sends results to ML Metadata for lineage to training parameters, dataset version, and code version.

Vertex Explainable AI provides feature-based explanations where feature attributions show which inputs moved each tabular prediction away from its baseline. Sampled Shapley approximates exact Shapley values by sampling permutation paths, so increasing path count such as sampledShapleyAttribution.pathCount: 6 increases precision and low counts produce unstable explanations. The explanationSpec defines method and precision in parameters and input-to-baseline mapping in metadata, while BigQuery ML separates local per-row functions like ML.EXPLAIN_PREDICT from global functions like ML.GLOBAL_EXPLAIN that average absolute attribution to reveal systematic reliance on a sensitive proxy.

Monitoring, testing, and troubleshooting AI solutions is the closed loop that samples live traffic, compares it to baselines, alerts on change, and recovers through retraining or rollback. Vertex AI Model Monitoring compares Endpoint request-response payloads to training and production baselines, evaluates per-feature thresholds, and exports violations to Cloud Run (opens in a new tab) telemetry, Cloud Monitoring, and Pub/Sub. The Endpoint keeps serving on one side of the trust boundary while monitoring and remediation operate on the observability side.

2.1Training-Serving Skew, Inference Drift, and Quality Thresholds

Training-serving skew, inference drift, and quality degradation are three drift classes with different baselines and fixes. Training-serving skew compares live serving features to the training dataset that produced the deployed model and catches pipeline bugs such as sending a product name instead of a product code. Inference drift compares recent production features and predictions to an earlier production window and catches changing world behavior when training data is no longer representative. Response quality degradation covers prediction distribution shift, confidence decay, ground-truth accuracy drop, and attribution drift.

Each monitored feature has its own threshold policy owned by the monitoring job, defining tolerable statistical distance, persistence duration, and whether a breach alerts or only logs. Separate thresholds are needed for semantic input-output drift in prompts, embeddings, response length, and toxicity; feature distribution skew for structured tabular columns; and response quality for usefulness and safety. The detection lifecycle runs in order: log and sample payloads, compute window statistics, compare to baselines, evaluate thresholds in Cloud Monitoring, publish job ID, feature, distance, and window to Pub/Sub, then trigger triage, canary comparison, rollback, or continuous training with lineage back to Vertex ML Metadata.

2.2Skew and Drift Detection with BigQuery ML and Validation Functions

Skew and drift detection with BigQuery ML and validation functions turns distribution comparison into SQL checks without separate infrastructure. Training statistics saved at model creation become the fixed baseline that serving statistics flow into for comparison. Basic checks divide by purpose:

• ML.DESCRIBE_DATA computes descriptive statistics to verify training or serving data stays within expected parameters.
• ML.VALIDATE_DATA_SKEW compares serving statistics to saved training statistics to flag anomalous differences.
• ML.VALIDATE_DATA_DRIFT compares two serving datasets, such as current data versus a snapshot or ML.FEATURES_AT_TIME output.
• ML.TFDV_DESCRIBE generates fine-grained statistics like TensorFlow tfdv.generate_statistics_from_csv.
• ML.TFDV_VALIDATE compares two JSON statistics objects in skew mode for training-to-serving or drift mode for serving-to-serving.

Statistics for skew are computed only for serving columns that match training columns, which improves performance and lowers cost. Advanced use stores TensorFlow Data Validation (TFDV) statistics as JSON protocol buffers with CURRENT_TIMESTAMP(), anchors comparison to a golden dataset and TFDV schema, and writes batch metrics to Cloud Storage as metrics/skew/feature_skew.json. The skew workflow runs describe, train, validate skew, then investigate and retrain, while drift compares consecutive serving spans before datasets diverge too far to fix.

2.3Feature Attribution Drift and Concept Drift

Feature attribution drift and concept drift detect when model logic or the world changed even when raw inputs look stable. Vertex Explainable AI attribution scores measure per-feature contribution for one instance, where a prediction request returns only the value and an explanation request returns prediction plus signed attributions that sum to the prediction score. Because attributions are numeric, standard univariate drift applies even to embeddings and time series. Interpretation tracks top features first, sums related dimensions such as grade, sub_grade, and purpose, and accounts for interactions that shift attribution without changing marginal distributions.

Concept drift means P(y|X) changed so the same features need a different label, which cannot be confirmed from inputs alone. Predictions logged to serving_predict must be joined to delayed true labels, where sustained drops in accuracy, precision, recall, F1, AUC-ROC, or higher log loss signal drift. Attribution skew uses a training baseline URI in Cloud Storage or BigQuery, while attribution prediction drift uses an earlier production window. Custom models need ExplanationSpec.ExplanationParameters populated and a known schema, or the job stays pending until it parses the first 1000 requests. Remediation promotes retraining through multi-agent AI system (opens in a new tab) pipelines and Model Registry with canary release and rollback to the last stable version.

2.4Generative AI Evaluation and Production Observability

Generative AI evaluation and production observability combine output-quality scoring with serving telemetry from the user's perspective. The Gen AI evaluation service scores variable outputs that accuracy alone cannot judge, with responses flowing from endpoints into scheduled jobs and breaches triggering rollback, retraining, or prompt revision. Standardized pipelines use the same datasets, metrics, and raters on every version, with data flowing from BigQuery or Cloud Storage into evaluation, scores into ML Metadata, and drift signals into Pipelines.

The evaluation dataset must reflect production distribution, supported by golden baselines, LLM judges, user feedback, and grounding checks for Identity and Access Management (IAM) (opens in a new tab) and RAG factuality. Serving telemetry follows latency including time to first token (TTFT), traffic including queries per second and token rate, error rate including low-confidence predictions, and saturation of CPU, GPU, and TPU. Reliability goals become service-level objectives such as 99.9% successful responses, 95th percentile latency below 300 ms, TTFT below 500 ms, and harmful output below 0.1%. RAG signals separate retrieval failure from generation failure through context relevance, groundedness, hallucination rate, toxicity, vector-database throughput, and user ratings, with vLLM and Triton telemetry via Managed Service for Prometheus, Cloud Trace, and Cloud Profiler.

2.5Safety Testing, Staged Release, and Remediation

Safety testing, staged release, and remediation block harmful or manipulated generative behavior and prove recovery works. Systematic red teaming probes for prompt injection, jailbreaking, leakage, and hallucination under Vertex AI safety thresholds. Inline responsible AI filtering sits between model and user: the application generates a response, saves prompts and responses to BigQuery for offline review, passes the response through filters, then sends only the screened response to the frontend. Offline alignment uses Pub/Sub to trigger a Cloud Run function that reads prompts from AlloyDB, scores factual accuracy and relevance, and loads scores to BigQuery. Injection detection for computer use agents that read screenshots is disabled by default and must be enabled with enable_prompt_injection_detection.

Staged validation proves an artifact is fit before broad exposure. Validation starts in staging to verify runtime dependencies, satisficing metrics for latency and size, and edge cases such as all features set to None, including supervised tuning checks with assess_tuning_validity() and assess_tuning_resources(). Live release follows fixed order: smoke-test the service API with one instance, canary-test with a small live stream, verify rollback to the prior Model version, then run A/B or Multi-Armed Bandit experiments before full promotion. Execution errors map to action with HTTP codes: 400 for bad schema, 403 for missing role, 408 for lock timeout, 429 for quota exhaustion with backoff, and 503 for backend overload. Agent Platform (opens in a new tab) Pipelines, traffic splitting, and Model Registry lineage link code, data, hyperparameters, checkpoints, and results so regression traces to one change.

Key Facts

• VPC Service Controls denies data flows from Vertex AI endpoints, Model Registry, BigQuery, and Cloud Storage to outside the perimeter by default, allowing them only through explicit ingress or egress rules.
• Sensitive Data Protection inspection combined with Model Armor data_protection_config and a DeidentifyTemplate enforces tokenization, masking, or redaction on prompts before the model and on responses after the model.
• A general execute_sql tool allows injection-driven cross-tenant reads, while a narrow custom tool enforces tenant isolation by injecting the authenticated user ID in backend code outside agent control.
• Training-serving skew compares live serving data to training statistics, while data drift compares earlier serving data to later serving data and concept drift compares predictions to delayed ground-truth labels.
• Vertex Explainable AI attributes tabular predictions to input features, while trace logs explain LLM agents by recording the ordered process of model calls, tool uses, and reasoning steps.