Prepare and test your skills
Prepare and test your skills
Post-hoc explainability techniques explain a trained black-box model without changing the model or its predictions. BigQuery ML (opens in a new tab) adds explanations through SQL functions, and Vertex Explainable AI adds explanations through online and batch prediction services that return per-feature attribution scores alongside each prediction. Both paths preserve model performance while producing feature importance scores and local explanations for governance and regulatory review.
ML.EXPLAIN_PREDICT is an extended version of ML.PREDICT that returns predictions plus per-row explanations for individual instances. The function outputs all input feature columns and then adds attribution columns that quantify how much each feature contributed to that row's predicted outcome. For logistic regression models, the attributions are Shapley values, which measure relative feature attribution for each feature.
A typical local explanation query controls output size with top_k_features and reads results in order:
dataframe = 'evaluation'.ML.EXPLAIN_PREDICT(MODEL census.census_model, input_query, STRUCT(3 AS top_k_features)).Because top_k_features was set to 3, the query returns the top three attributions for each row. The attributions are ordered by absolute attribution value in descending order, so the most influential features for that specific instance appear first whether their effect was positive or negative.
ML.GLOBAL_EXPLAIN explains the model as a whole by identifying which features are most important for determining the target across the dataset. While ML.EXPLAIN_PREDICT answers why the model made one specific prediction for one row, ML.GLOBAL_EXPLAIN aggregates importance to answer which features drive decisions such as income bracket overall. Global explanations are used to validate model behavior, compare candidate features, and document transparency for stakeholders without retraining the model.
ML.EXPLAIN_FORECAST forecasts future time-series values and returns all separate components of the time series. The function uses the same horizon and confidence controls as ML.FORECAST, for example STRUCT(30 AS horizon, 0.8 AS confidence_level) forecasts 30 future time points with an 80% confidence prediction interval. Output rows are ordered chronologically by the time_series_timestamp column value.
The explanation output decomposes each timestamp into interpretable components. History and forecast rows include time_series_data, time_series_adjusted_data, standard_error, prediction_interval_lower_bound, and prediction_interval_upper_bound, plus components such as:
trend, holiday_effect, spikes_and_dips, step_changes, residualseasonal_period_yearly, seasonal_period_quarterly, seasonal_period_monthly, seasonal_period_weekly, seasonal_period_dailyIn BigQuery DataFrames, the same capability is available through predict_explain, for example model.predict_explain(horizon=30, confidence_level=0.8), which returns forecast data plus the component columns. Data flows from historical time-series tables into the forecasting model, and component explanations flow back to the analyst alongside the forecast.
Exam tip: Use ML.PREDICT when only predictions are needed, and use ML.EXPLAIN_PREDICT with STRUCT(n AS top_k_features) when each row also needs its top-n attributions ordered by absolute Shapley value.
Vertex Explainable AI provides post-hoc explanations for deployed models through online and batch prediction. For online inference, the client calls endpoint.explain(instances=[{...}, {...}]) with the same instance format used for online prediction, and the endpoint returns a prediction plus an explanation for each instance. For batch use cases, Vertex AI Batch Prediction runs the same explained prediction over a stored input set, which supports audit and compliance workflows on large volumes.
Each Vertex explanation compares a baseline to the current instance and distributes responsibility across features. baselineOutputValue is the reference output value used as the comparison point, while instanceOutputValue is the model output value for the explained instance. The featureAttributions map assigns each input feature, such as store_id, dept_id, item_id, date, and sales, to its numeric attribution score for that instance. For forecasting models, Vertex also returns context_columns, horizon_columns, and horizon_weights, where horizon weights are the feature importance weights associated with each horizon column.
The explanation functions split by whether the question is local, global, or temporal. Local scope explains one row, global scope explains the whole model, and temporal scope explains how time components produced a forecast:
| Function | Scope | Use when |
|---|---|---|
| ML.EXPLAIN_PREDICT | Single prediction | The question is why the model predicted this outcome for this row |
| ML.GLOBAL_EXPLAIN | Entire model | The question is which features are most important overall |
| ML.EXPLAIN_FORECAST | Time series | The question is how trend, seasonality, holidays, and residuals produced the forecast |
All three approaches preserve the trained model and add transparency alongside predictions, so local explanations and importance scores can be generated for governance while model performance remains unchanged.
Trade-off analysis between model complexity and explainability weighs accuracy gains from larger, more opaque architectures against higher training cost, longer search time, and harder-to-trace decisions. Agent Platform Neural Architecture Search illustrates this trade-off directly, because expanding the search space and model size can improve accuracy but increases compute cost, trial duration, and difficulty in explaining why one architecture was selected. Managing the trade-off therefore requires controlling baseline training cost, using scaled-down proxy tasks, and budgeting total and parallel trials before launching a full search.
Agent Platform Neural Architecture Search requires an optimized baseline model before search, because every inefficiency in training is multiplied across thousands of trials. Teams separate block design from scaling by searching first with a scaled-down model and then running a separate search to scale the model up, which keeps proxy-task runtime much lower while still exploring architectural choices. If certain architecture blocks are independent of the search space, teams load the same pretrained checkpoints for those blocks across all trials, for example using a fixed encoder while only the decoder is searched, so training is faster without introducing bias across candidates.
Training efficiency in Agent Platform Neural Architecture Search depends on data, compute, and memory choices that interact with each other:
After these levers, batch size remains a direct complexity trade-off because higher utilization lowers cost per trial, while excessive size causes trial failures that waste the search budget.
Search budgeting in Agent Platform Neural Architecture Search balances convergence confidence against days of compute. Convergence typically requires on the order of approximately 2,000 trials, although the exact number varies with search-space size and cannot be known beforehand. The recommended practice is to set --max_nas_trial very high, approximately 5,000 to 10,000, then cancel the search job earlier if the search plot shows convergence.
The full search runs as batch processing controlled by --max_parallel_nas_trial, which sets how many trials run in parallel and critically reduces overall runtime. Expected search duration follows days-required-for-search = (trials-to-converge / max-parallel-nas-trial) * (avg-trial-duration-in-hours / 24). Initial planning can use 3,000 as a rough upper bound for trials-to-converge and 2 hours as a rough upper bound for average proxy-task duration. A previous search can be resumed once with the search_resume command, but a resumed search cannot itself be resumed again.
Exam tip: Set --max_nas_trial to 5,000–10,000 and cancel early on convergence, rather than guessing the exact 2,000-trial convergence point; a search job can be resumed only once.
Resource choice per trial in Agent Platform Neural Architecture Search trades per-trial speed against startup overhead and total throughput. Using fewer GPUs per trial reduces startup time and is more efficient for proxy tasks, even if an individual trial could theoretically run faster with more accelerators:
| Trial resource option | Startup cost | Choose when |
|---|---|---|
| 2 GPUs per trial | About 5 minutes to start | Running many short proxy-task trials where startup overhead dominates |
| 8 GPUs per trial | About 20 minutes to start | Only when a single trial needs the extra parallelism enough to offset the longer startup |
Choosing fewer GPUs per trial preserves parallel capacity for --max_parallel_nas_trial, so more architectures are evaluated per day. This is the same logic as the broader complexity-explainability decision because accepting a smaller, more tractable unit of work per trial allows more candidates to be explored, compared, and explained within a fixed budget.
Designing model architectures with built-in interpretability means choosing model families, training paths, and serving configurations where explanation is part of the structure rather than an afterthought. On Google Cloud, built-in interpretability is achieved by selecting architectures that support Vertex Explainable AI, using BigQuery ML (opens in a new tab) explanation functions, and enforcing operational constraints so transparency holds under production latency requirements. The design balances optimizing metrics that measure predictive effectiveness against satisficing metrics that define hard operational limits.
Balancing performance and interpretability starts with two distinct metric types that constrain architecture choice from experimentation through production. Optimizing metrics measure predictive effectiveness and improve with higher values, including accuracy and f-measure for classification, mean absolute percentage error for regression and forecasting, discounted cumulative gain for ranking, and perplexity and BLEU scores for language models. Satisficing metrics measure operational constraints that reject any model that violates a threshold, such as prediction latency set to 200 milliseconds or model size for mobile and embedded deployment. During experimentation, the architecture is developed, trained, evaluated, and debugged to improve optimizing metrics without violating satisficing thresholds, with fairness assessed by achieving similar effectiveness across data slices such as customer demographics.
Production latency turns the satisficing threshold into service-level objectives (SLOs) that the interpretable architecture must meet on every request. Examples include 95th percentile inference latency below 300 ms and time to first token (TTFT) below 500 ms for 99% of requests for large language models (LLMs). Because Vertex Explainable AI incurs minor additional cost and compute for attribution, the choice of explanation method directly affects whether the endpoint can still meet its latency SLO. The architecture therefore depends on both the model and its explanation path, and the serving platform must be sized and tested for prediction correctness, latency, throughput, and API function before traffic is shifted.
Intrinsically explainable model families on Gemini Enterprise Agent Platform (Agent Platform) are supervised tabular models that natively support feature attributions at serving time. Vertex Explainable AI details the importance of each input feature for a prediction, which helps verify that the features driving predictions align with domain knowledge and ethical guidelines. Support is divided by architecture family, with deep networks using one attribution method and tree-based ensembles using another:
| Architecture family | Model types covered | Explainable AI method |
|---|---|---|
| Deep neural networks | dnn_classifier, dnn_regressor, dnn_linear_combined_classifier, dnn_linear_combined_regressor | Integrated gradients |
| Tree ensembles | boosted_tree_classifier, boosted_tree_regressor, random_forest_classifier, random_forest_regressor | Sampled shapley |
Choose the deep network family when the task needs learned nonlinear representations and Integrated gradients can trace output back to inputs, and choose the tree ensemble family when the task needs partitioned decision logic where Sampled shapley can summarize feature contributions. Vertex Explainable AI also supports custom-trained models based on tabular and image data, extending the same attribution principle beyond the listed exportable models. For interactive analysis during development, a Vertex AI Workbench instance provides the What-If Tool (WIT) for model-agnostic bias exploration and the Language Interpretability Tool (LIT) for visual exploration of natural language processing behavior, with WIT integrating with TensorBoard.
Exam tip: Integrated gradients applies to DNN classifiers and regressors, while Sampled shapley applies to boosted-tree and random-forest classifiers and regressors.
BigQuery ML provides built-in SQL explanation functions that extend prediction and weight functions so interpretability is queried directly from the warehouse. ML.EXPLAIN_PREDICT is an extended version of ML.PREDICT that returns predictions with per-row feature attributions, and ML.GLOBAL_EXPLAIN returns global explainability by taking the mean absolute attribution that each feature receives across all rows in the evaluation dataset. ML.EXPLAIN_FORECAST is an extended version of ML.FORECAST for time-series models, and ML.ADVANCED_WEIGHTS is an extended version of ML.WEIGHTS for inspecting learned weights. Data flows from BigQuery tables through the training statement into a registered model, and then explanation flows back from the model to the analyst through these SQL functions without moving data to a separate serving system.
Serving explanations under production constraints means deploying an explainable model to an Agent Platform endpoint with logging and monitoring wired to the same release. For tabular models deployed on Agent Platform endpoints, the endpoint generates feature attributions that indicate the input features that contributed most to each prediction. Explanations then flow from the endpoint to two destinations: to monitoring dashboards for developers and to application interfaces for end users when understanding the reasoning is important for trust or decision-making. The deployment depends on Model Registry, which stores the model version with metadata including training parameters, evaluation metrics, dataset version with lineage, code version, and documented limitations, intended use cases, and known biases.
Deploying a BigQuery ML model with explainability follows an ordered workflow:
n1-standard-2, to meet latency and cost constraints.Only after deployment, online explanations can be requested from the deployed model alongside predictions. Continuous monitoring then watches for shifts that indicate emerging bias or unfairness, including data drift in input distribution, prediction drift in outputs, and spikes in low-confidence predictions, with results fed back into re-evaluation or retraining cycles.
Process transparency for generative AI systems explains the process that an agent followed because token-level attribution is relatively challenging for large language models. Trace data flows from the agent runtime to trace logs, which record the sequence of model calls, tool calls, and retrieved context that produced the output. In retrieval-augmented generation (RAG) applications, retrieved information flows from the vector store and source documents into the prompt, and citations flow from the application back to the user to ground each claim in its source. Prompt engineering guides the model to provide explanations or show its reasoning steps, and grounding provides a second layer of source verification at the user level.
Evaluation and governance close the loop between transparency and control. The Gen AI evaluation service computes metrics for quality, factuality, safety, and style adherence, including a grounding metric that checks factuality against provided source text during verification. Model Monitoring on Agent Platform tracks output characteristics such as toxicity, coherence, hallucination rates, and user-feedback ratings, while audit trails track who accessed specific models or tools, what data was used for fine-tuning or prompting, and what queries were sent to production endpoints. Together, trace logs, citations, evaluations, and registry metadata preserve the boundary between the model control plane that manages versions and policies and the data plane that carries prompts, retrieved context, and generated outputs.
ML.EXPLAIN_PREDICT explains a single prediction for one row, ML.GLOBAL_EXPLAIN explains the entire model by identifying which features are most important overall, and ML.EXPLAIN_FORECAST explains time-series forecasts by decomposing them into components such as trend, seasonality, holiday effects, and residuals. ML.EXPLAIN_PREDICT returns per-row predictions with top-n attributions ordered by absolute attribution value, while ML.GLOBAL_EXPLAIN aggregates importance across the dataset and ML.EXPLAIN_FORECAST returns forecast values with component columns for each timestamp.
Vertex Explainable AI returns a prediction plus an explanation for each instance through online explain calls and Vertex AI Batch Prediction for stored input sets. Each explanation includes baselineOutputValue as the reference output value, instanceOutputValue as the model output for the explained instance, and a featureAttributions map that assigns each input feature to its numeric attribution score for that instance.
Use Integrated gradients for deep neural network classifiers and regressors, including dnn_classifier, dnn_regressor, dnn_linear_combined_classifier, and dnn_linear_combined_regressor. Use Sampled shapley for tree ensemble classifiers and regressors, including boosted_tree_classifier, boosted_tree_regressor, random_forest_classifier, and random_forest_regressor.
Last updated