Low-Code AI Solutions with BigQuery ML

Developing ML models using BigQuery ML or AutoML on Gemini Enterprise Agent Platform is low-code training where SQL, automated search, and managed pipelines replace custom code. BigQuery ML (opens in a new tab) trains models with SQL where warehouse data already lives, Gemini Enterprise Agent Platform (opens in a new tab) AutoML automates architecture search and tuning, and custom training handles the remaining cases. Data flows from storage to training through the BigQuery Storage API, BigQuery I/O connector, or Cloud Storage sharded reads, and the correct path preserves data locality. The choice follows a fixed order: confirm BigQuery locality and SQL fit first, then AutoML format and latency fit, and only then use custom training.

1.1Choosing the Training Environment for Data Location and Latency

The training environment is the system where data, code, and serving meet, and choosing a training method (opens in a new tab) depends on where data lives and what latency is allowed. BigQuery ML keeps tabular workflows in one BigQuery system with SQL training, AutoML automates architecture choice and provisioning for supported modalities, and custom training on Agent Platform runs user-owned routines serverless with tuning, TensorBoard, and monitoring. BigQuery ML can also invoke AutoML tabular search from SQL without moving data, while large tables use partitioning for cost and performance and materialized tables are preferred over views or subqueries for training speed.

1.2Mapping Business Problems to BigQuery ML Model Types

Mapping business problems to BigQuery ML model types means selecting the objective first and then supplying the columns that objective requires. Classification predicts a category, regression predicts a numeric value, forecasting predicts a future value, and clustering assigns group membership without a label. Hyperparameter tuning is enabled with the NUM_TRIALS option, where available arguments vary by model type. Classification and regression require INPUT_LABEL_COLS, forecasting with ARIMA_PLUS requires TIME_SERIES_TIMESTAMP_COL and TIME_SERIES_DATA_COL, k-means clustering needs no label, matrix factorization for recommendation requires RATING_COL, USER_COL, and ITEM_COL, and principal component analysis (PCA) and autoencoder compress high-dimensional inputs.

1.3Configuring AutoML Tabular and Forecasting Jobs

AutoML tabular and forecasting jobs bind a managed dataset to a business objective and then search, train, and ensemble candidates into one artifact. An AutoMLTabularTrainingJob uses target_column, optimization_prediction_type such as regression, and optimization_objective such as minimize-rmse, with built-in preprocessing handling transformations automatically. Binary classification needs less data than multi-class or regression, and the search considers neural networks and boosted trees before ensembling top performers. Tabular Workflow for End-to-End AutoML exposes data splitting, feature engineering, architecture search, training, ensembling, and distillation in order, supports multiple terabytes and up to 1000 columns, and caps compute with budget_milli_node_hours and splits such as 0.6, 0.2, and 0.2.

Tabular Workflow for Forecasting supports up to 1 TB and 200 columns, and unstructured inputs should use files of at least 100 MB sharded between 100 and 10,000 shards.

1.4Engineering Features Inside BigQuery ML

Feature engineering in BigQuery ML keeps transformation logic inside the warehouse so raw tables flow directly to training without an external cluster. The TRANSFORM clause in CREATE MODEL defines custom preprocessing during training, and learned statistics are automatically reused during prediction with ML.PREDICT. Automatic preprocessing is deterministic by type: numeric NULL values become the column mean, one-hot and multi-hot NULL values map to an additional category while unseen prediction categories get weight 0, and each field of a STRUCT follows its type rule. Reusable supply through Knowledge Catalog discovery, BigQuery sharing, Agent Platform Feature Store, object tables, and Vector Search indexes avoids copying data for every model.

1.5Creating SQL Features with TRANSFORM Functions

The TRANSFORM execution model defines how custom SQL moves from training to prediction in BigQuery ML. Manual functions can be used inside TRANSFORM and outside it for ad-hoc preparation, and every ML analytic function requires an empty OVER() clause to signal collection across all rows. ML.TRANSFORM returns the actual preprocessed rows that enter the model, while temporal decomposition, ML.POLYNOMIAL_EXPAND, and bucketization let linear models learn seasonality and curves. ML.BUCKETIZE uses caller-supplied boundaries while ML.QUANTILE_BUCKETIZE learns balanced-count buckets, and categorical work uses ML.ONE_HOT_ENCODER, ML.MULTI_HOT_ENCODER, ML.LABEL_ENCODER, ML.HASH_BUCKETIZE, and ML.FEATURE_CROSS for joint effects such as device and hour-bucket together. Boosted tree and random forest models skip standardization, while k-means standardizes only when STANDARDIZE_FEATURES is enabled.

1.6Selecting Features from Weights and Exported Trees

Automated feature selection in BigQuery ML uses saved model weights to rank which inputs most influence predictions. Linear and logistic regression expose coefficients through ML.WEIGHTS, k-means exposes centroids through ML.CENTROIDS, matrix factorization exposes latent factors through ML.WEIGHTS, PCA exposes eigenvectors through ML.PRINCIPAL_COMPONENTS and ML.PRINCIPAL_COMPONENT_INFO, and ARIMA_PLUS exposes trend coefficients through ML.ARIMA_COEFFICIENTS. High-magnitude coefficients and frequent split features are retained, while near-zero or unused features are candidates for removal. Boosted tree, random forest, deep neural network (DNN), wide-and-deep, and AutoML Tables models have no in-warehouse weight function and require export for inspection:

1. Export: Run EXPORT MODEL from BigQuery ML to a Cloud Storage bucket.
2. Download: Download the model.bst file from the bucket locally.
3. Upload: Upload model.bst to the Files area of a Colab notebook.
4. Plot: Load with xgb.Booster and call xgb.plot_tree per iteration to render trees.

Exported trees use encoded identifiers such as f0, which map to real columns through assets/model_metadata.json, while categorical split values map by position to vocabulary files in assets/.

1.7Invoking Remote Models and Generating Predictions

A remote model in BigQuery ML is a model object that points to a Vertex AI foundation model or deployed endpoint. The remote model depends on a Cloud Resource connection, which provides a dedicated service account as the identity for calls from BigQuery to Vertex AI and Cloud Storage. The trust boundary sits between BigQuery on one side and Vertex AI APIs and Cloud Storage on the other, and the connection account must hold the Vertex AI User role plus bucket read access. Creation follows a fixed order: first create the connection in the same project and region as the dataset, then grant the service account access, then run CREATE MODEL with REMOTE WITH CONNECTION naming the connection and ENDPOINT, and then query with ML.PREDICT, ML.GENERATE_TEXT, or ML.GENERATE_EMBEDDING.

1.8Generating Text, Embeddings, and Batch Scores

ML.GENERATE_TEXT sends prompt columns to a large language model and returns generated text as structured rows with text, safety attributes, and metadata. Common practices include instructions plus few-shot examples, constraining output to JSON for parsing into STRUCT fields, and batching prompts for summarization, classification, extraction, and question answering. ML.GENERATE_EMBEDDING converts text or multimodal content into dense vectors that preserve semantic distance, where one vector flows back per row for storage, vector-search indexing, and joins. An object table maps Cloud Storage files to queryable rows without copying bytes, so URIs plus prompts feed directly into generative functions, while image bytes flow through ML.DECODE_IMAGE, then ML.CONVERT_COLOR_SPACE and ML.CONVERT_IMAGE_TYPE, and only then into ML.PREDICT. Batch scoring with ML.PREDICT, ML.FORECAST, ML.RECOMMEND, and ML.DETECT_ANOMALIES scores stored tables in bulk, where row-level failures return NULL for that row with a default threshold of 1.0 and LIMIT applies after the model processes a batch.

1.9Training and Operating AutoML Pipelines on Agent Platform

Training with AutoML on Agent Platform automates architecture search, tuning, and provisioning for tabular, forecasting, image, text, and video problems. The AutoML TrainingPipeline resource always executes training and can optionally export data from a Dataset, upload the resulting Model, and evaluate the Model. Data flows from the Dataset into training, from training to Model Registry for versioning, and from the registry to an endpoint or batch service. The same managed dataset can train both custom and AutoML models for direct comparison, with Cloud Storage for objects, BigQuery for tabular analysis, and NetApp Volumes or Filestore for shared files. Pipeline stages run in order through StatsAndExampleGenOp, TransformOp, SplitMaterializedDataOp, TrainingConfiguratorAndValidatorOp, Stage1TunerOp and CvTrainerOp, and EnsembleOp and FinalizerOp, with metadata linking every step.

Version control keeps code, data, features, and binaries reproducible and rollback-ready. Model Registry versions model binaries and metadata, Artifact Registry versions containers and pipeline components, and Feature Store versions feature data with BigQuery as the offline store. Automated deployment uses Agent Platform Pipelines with scheduled runs and Cloud Scheduler, plus Cloud Build validation and Dataflow feature validation. After deployment, Model Monitoring detects drift, performance loss, and anomalies, and versioned lineage enables rollback or retraining.

1.10Fine-Tuning Gemini Models with BigQuery Data

Fine-tuning with Gemini in BigQuery adapts a Gemini foundation model to an enterprise task with supervised tuning. The job reads a labeled JSONL dataset from Cloud Storage, learns adapter parameters on the frozen base model, and outputs a tuned model with its own resource, endpoint, and checkpoints. Data flows from operational sources into BigQuery materialized tables, where built-in transformations preprocess it before CREATE MODEL runs. For maximum speed, training data is stored as materialized partitioned tables rather than views or subqueries, and quality covers accuracy, relevance, diversity of phrasing, and alignment of labels with quality, relevance, and safety.

Supported bases include Gemini 3.5 Flash, Gemini 3.1 Flash-Lite, Gemini 2.5 Pro, Gemini 2.5 Flash, and Gemini 2.5 Flash-Lite. Each job selects one base model, a display name up to 128 characters, a tuning region, and training plus optional validation datasets. User data stays in the tuning region while compute can offload to other US or EU regions, and limits include 131,072 input and output tokens per example, 1 GB JSONL size, 10M text-only or 300K multimodal examples, validation capped at 5000 examples, tuning in us-central1 and europe-west4, and no customer-managed encryption keys (CMEK). Epoch count, adapter size with values 1, 2, 4, 8, and 16, and learning rate multiplier with default 1 trade learning power against token cost, overfitting, and instability. Managing BigQuery ML models in Agent Platform (opens in a new tab) uses Vertex AI Model Registry for versioning and online prediction, where each BigQuery ML model registers to only one model ID and iterations use aliases.

Building AI solutions using Google Cloud AI APIs or foundational models is matching a task to Document AI, Vision, Translate, Gemini, Imagen, Veo, or a Model Garden model and then serving it reliably. The candidate model receives prompts and produces responses, the Gen AI evaluation service measures those responses, and the serving configuration determines whether latency, throughput, cost, and accuracy targets are met. A model is an optimal fit only when task type, input and output handling, and serving compatibility all align.

2.1Selecting Models from Model Garden by Task and Compatibility

Model Garden selection translates a use case into objective, input shape, and expected output, then checks them against model attributes. Prompt datasets live in BigQuery or Cloud Storage and batch outputs return as BigQuery tables or Cloud Storage URIs. Pipeline tasks organize the match: text-generation for open-ended generation, text2text-generation for summarization and translation, text-to-image for image generation, feature-extraction for embeddings, sentence-similarity for semantic comparison, and image-text-to-text for vision-language tasks. Serving compatibility is a hard constraint because the vLLM container supports only text-generation and image-text-to-text models, so other tasks cannot use the default one-click deployment path and must fall back to a custom notebook, feature request, or future addition.

2.2Validating Model Fit with Gen AI Evaluation

Validating model fit with the Gen AI evaluation service means defining criteria before selection and measuring candidate outputs against them. Criteria are dimensions such as conciseness, relevance, and correctness, while a metric is a single score for output against those criteria. Data moves in order: first input prompts flow from the evaluation dataset to each candidate model, then generated responses flow to the judge model for scoring, and finally row judgments flow back for aggregation with chain-of-thought explanations. Gemini is the judge for most uses, with MetricX or COMET for translation.

Within model-based metrics, pointwise scoring rates one response on a scale such as 0 to 5, while pairwise comparison picks the better of two models and is only supported with Gemini as judge. Pairwise checks use AutoraterConfig with flip_enabled=True to reduce position bias and multi-sampling with sampling_count from 1 to 32 and default 4.

2.3Choosing Online, Batch, and Private Serving Paths

Choosing a serving path decides how features reach the model and how predictions return to the application. Vertex AI online prediction receives instances and returns predictions in the response for near real-time use, while batch prediction scores data from a data lake on a schedule. Online callers first fetch features from a low-latency store such as the Feature Store online serving API and then call the endpoint, while batch callers fetch from the data lake or batch serving API and streaming ingestion makes updates available for online serving within seconds. Custom routines customize request handling, multiple models can share one endpoint for gradual replacement, and separate endpoints isolate testing from production.

For online prediction, set automatic scaling with minimum and maximum nodes, with a minimum of two nodes to support a high availability service level agreement (SLA).

2.4Controlling Cost, Scaling, and Accelerator Capacity

Controlling cost and capacity for Model Garden serving means matching CPU virtual machine (VM) types and graphics processing unit (GPU) types to model size and latency targets, then forecasting demand. Cloud Monitoring collects utilization and serving signals, Cloud Logging collects logs, and analysis in BigQuery or Looker Studio forecasts GPU and tensor processing unit (TPU) needs. The observability dashboard exposes model queries per second (QPS), token throughput, and first token latencies, and load testing with Apache JMeter or LoadView validates behavior under high load. Cost optimization for AI and ML (opens in a new tab) trades commitment for flexibility through Committed use discounts (CUDs) for stable 24x7 inference, Spot VMs for fault-tolerant evaluation, and Dynamic Workload Scheduler (DWS) with Flex-Start for batch training up to seven days and Calendar mode to reserve H100 and H200 GPUs and Trillium TPUs up to three months. GKE Inference Gateway distributes traffic with model-aware routing, while vLLM dynamically uses TPUs and GPUs and Ray on Vertex AI abstracts VMs and containers.

2.5Grounding Selected Models with Enterprise Data

Grounding selected models with enterprise data connects a generative model to retrieved documents so responses reflect current facts. RAG Engine uses Vector Search 2.0 to store vector representations and retrieve by semantic similarity, while managing collections that remain visible in the project. RAG Engine corpora backed by Vector Search 2.0 are available only in us-central1, support Virtual Private Cloud Service Controls (VPC-SC) and customer-managed encryption keys (CMEK) handling at the service level, but do not support data residency and Access Transparency controls. Pre-trained models are ready for generalized data, while user-trained models use AutoML or custom training for unique labels, with Vision supporting AutoML vision models created in October 2022 or later.

Choose RagManagedVertexVectorSearch for latest features with automated management, and choose RagManagedDb only when CMEK is a strict requirement.

2.6Monitoring API Health and Designing for Scale

Monitoring deployment health and business thresholds shows whether a Document AI, Vision, or Translate integration stays healthy under real traffic. Infrastructure can include Compute Engine instances, Google Kubernetes Engine (GKE) clusters, and TPUs and GPUs, with Active Assist recommendations for optimization. Release state flows from Cloud Deploy to Cloud Monitoring for visibility of canary releases and rollbacks, while Security Command Center monitors threats separately from performance. The observability system combines Cloud Logging for logs, Cloud Monitoring for metrics and PagerDuty or Slack alerts, Error Reporting for production exceptions, Cloud Trace for distributed bottlenecks, and Cloud Profiler for CPU or memory bottlenecks.

Designing API infrastructure for scale ensures the selected API and its infrastructure handle growing volumes without bottlenecks. Planning starts with stakeholders to project growth, then uses Cloud Monitoring to analyze historical utilization and project future needs, validated by load testing before peaks expose bottlenecks. Teams must know quotas for Compute Engine, Agent Platform, and Cloud Storage, request increases with forecasting justification, and alert when usage approaches limits. Elastic capacity translates forecasts into cost controls through rightsizing, autoscaling, Spot VMs for fault-tolerant batch workloads, and documented peak-event drills with automation, accountability, and AI ethics checks.

2.7Decoupling and Transforming API Workloads at Scale

Decoupling API workloads at scale separates producers from processors so quota limits and latency do not stop the system. Pub/Sub is an asynchronous messaging service where producers send work to a topic and processors later pull from the topic. Data flows from producer to topic to processor, so bursts queue durably instead of overwhelming Document AI, Vision, or Translate, and both sides scale independently. A pull-based agent in each VM runs an ordered sequence: first listen by subscribing to assigned topics, then pull the next message only when idle, then start a pipeline in third-party software, then wait holding no additional work, and then advance to the next message after completion.

Choose Cloud Data Fusion when visual assembly and reusable plugins reduce effort, and choose Dataflow when custom windowing and unified Beam logic are needed.

2.8Optimizing API Cost, Latency, and Response Quality

Optimizing API cost, latency, and response quality shapes requests, reuses stored results, and gates outputs on quality scores. Decoupled serving keeps synchronous work minimal: bursts queue in Pub/Sub, Cloud Run functions (opens in a new tab) runs single-purpose serverless logic with automatic scaling, and Agent Platform with Vector Search lowers per-query latency for very large-scale retrieval. The serving path flows from model to BigQuery logging to responsible AI filtering to frontend delivery, so unsafe outputs are blocked without a second generation pass. Caching and quality evaluation separate operational reads from offline analysis: Cloud Storage holds raw objects, AlloyDB for PostgreSQL holds configuration and evaluation prompts for low-latency reads, and BigQuery holds responses and scores for analysis without re-invoking the model. Scores let high-confidence responses go directly to users while low-confidence responses go to human review or a second retrieval pass.

2.9Customizing Gemini with Supervised Fine-Tuning

Customizing Gemini with supervised fine-tuning (SFT) teaches a new skill by training on hundreds of labeled input-output examples. In Vertex AI, Gemini SFT is parameter-efficient tuning where base parameters stay frozen and only small adapter weights are learned, then combined with the original model for serving. SFT minimizes the difference between predictions and labels and is the correct choice when output can be clearly defined, deviates from general patterns, produces inconsistent results with prompting, is too complex to articulate, or needs shorter context for lower latency and cost. Dataset preparation uses a labeled JSONL file in Cloud Storage matching production distribution, with limits of 131,072 maximum input and output tokens, 1 GB maximum JSONL size, up to 10M text-only or 300K multimodal examples, and validation limited to 5000 examples.

SFT is particularly effective for exact behavior: classification that forces a single label such as business, summarization that enforces custom format such as #Person1 and #Person2 with personally identifiable information (PII) removed, extractive question answering that returns an exact substring such as Last Glacial Maximum, and chat control that constrains the assistant to a role such as shopkeeper. Related options use different mechanisms: Reinforcement Learning from Human Feedback (RLHF) and preference tuning learn subjective preferences after SFT, distillation trains a smaller student to mimic a teacher, and grounding connects outputs to sources without changing weights.

2.10Running and Evaluating Tuning Jobs

Running a tuning job in Agent Platform executes a pipeline that reads data from Cloud Storage, learns adapter parameters, and deploys the tuned model and endpoint in the tuning job region. Supported bases include Gemini 3.5 Flash, Gemini 3.1 Flash-Lite, Gemini 2.5 Pro, Gemini 2.5 Flash, and Gemini 2.5 Flash-Lite. User data stays in the tuning region while compute may offload to other US or EU regions, billed training tokens equal dataset tokens multiplied by epochs, and quota is a global concurrent-jobs limit shared across regions. Adapter sizes are 1, 2, 4, 8, and 16, with some models supporting only 1, 2, 4, and 8.

The console flow in Agent Platform Studio runs in order:

1. Open tuning: Go to Agent Platform Studio and click Create tuned model.
2. Set model details: Enter a name up to 128 characters, select base model and region.
3. Set hyperparameters: Enter epochs, adapter size, and learning rate multiplier with default 1.
4. Set checkpoints: Optionally enable Export last checkpoint only.
5. Set training data: Choose upload or existing Cloud Storage JSONL file.
6. Set validation data: Optionally enable validation with Cloud Storage path.
7. Start and monitor: Click Start Tuning until Status shows Succeeded.

Evaluation combines tuning metrics for learning progress with validation metrics for generalization. If validation degrades while training improves, reduce epochs, adapter size, or learning rate, improve validation coverage, or select an earlier checkpoint.

2.11Deploying Tuned and Grounded Endpoints

Deploying tuned and Model-as-a-Service endpoints places foundation models behind scalable inference APIs with retrieval and safety controls. Model Garden is the catalog for discovering models: Gemini handles text and multimodal prompts with Google Search grounding, Imagen generates images, Codey generates code, and text-embedding models generate vectors. Traffic flows from user to Cloud Load Balancing frontend to Cloud Run frontend to retrieval service to Vertex AI for embedding and generation. The retrieval service depends on Secret Manager for keys and Cloud SQL or AlloyDB for vector lookup, so credentials never live in code. Secure endpoints authenticate with Secret Manager credentials, encrypt AlloyDB by default with CMEK option, enforce SSL-only connections, and reduce exfiltration with VPC Service Controls. After generation and before frontend delivery, responsible AI filters screen responses, then prompts and logs flow to BigQuery for audit.

2.12Running Enterprise Ingestion and Grounded Serving

Running enterprise ingestion and grounded serving in Gemini Enterprise separates data preparation from query handling so new data becomes retrievable quickly. The ingestion pipeline runs in order: first data engineers upload raw data to a Cloud Storage bucket, then Cloud Storage publishes completion to a Pub/Sub topic, then the topic triggers a Cloud Run functions job that enriches data and writes JSON Lines (JSONL) metadata to a second bucket, then the function publishes completion to a second Pub/Sub topic, and then the topic triggers ingestion into the managed datastore, which parses, chunks, and embeds automatically. The serving subsystem runs in order: first a Cloud Run frontend receives the query, then the frontend forwards it to a centralized backend that builds search filters, then the backend sends the request to the Gemini Enterprise API endpoint, then Gemini Enterprise embeds the query, performs semantic search, augments the prompt, and generates the response, and then the response returns through the backend to the frontend. Quality evaluation runs asynchronously from Pub/Sub trigger to function configuration from AlloyDB to scoring for factual accuracy and relevance to persistence in BigQuery.

2.13Synthesizing Video and Serving Open Models

Synthesizing video with Veo and serving open models from Model Garden covers two specialized serving patterns. Veo is a long-running model where the request flows from client to Veo endpoint, the service starts an operation, and video flows to the Cloud Storage bucket in output_gcs_uri or returns inline as bytes. The client must poll until operation.done is true, then read generated_videos[0].video.uri. Veo 3 generates 4, 6, or 8 second clips with default 8, Veo 2 generates 5 to 8 seconds with default 8, and first-frame and last-frame images constrain start and end composition. A Veo image-to-video request builds a request with model such as veo-3.1-generate-001, a motion prompt, and a first-frame image, adds controls such as last_frame, aspect_ratio, output_gcs_uri, sampleCount from 1-4, and seed, starts the operation, and polls to completion. Serving open models trades overhead for control: MaaS keeps compute provider-side behind a shared API, Model Garden deployment provisions a dedicated endpoint with chosen GPUs or TPUs, and custom vLLM containers provide full throughput tuning with vLLM, SGLang, TensorRT-LLM, TEI, PyTorch Inference, or Hex-LLM for TPU-optimized Llama serving.

2.14Optimizing Gemini Applications for Cost, Latency, and Availability

Optimizing Gemini-based applications means aligning model choice, prompt design, inference mode, and resiliency so cost, latency, throughput, and availability targets are met together. Traffic flows from clients to global load balancing to healthy Agent Platform endpoints, while telemetry flows from serving infrastructure to monitoring and analysis tools that drive capacity decisions. Regional endpoints keep data in specific regions, while the global endpoint pools Google-managed capacity across regions. Each layer replicates independently so one zone loss does not stop serving: GKE regional clusters replicate across zones, Cloud Storage multi-regional buckets replicate datasets, Spanner provides globally consistent agent metadata, Cloud SQL read replicas provide transactional state, and vector databases remain multi-zonal so retrieval is not a single point of failure.

Selecting models, optimizing prompts, and caching context are the three primary levers for cost and latency. Model selection sets baseline price and capability by starting with the most cost-efficient model and moving up only until quality is met. Prompt optimization reduces tokens with short, direct prompts such as summarize in 2 sentences, while Vertex AI context caching stores repeated high-token content once and reuses it across requests to reduce billed input tokens and time-to-first-token. Choosing inference modes decides how compute is allocated: Provisioned Throughput reserves guaranteed throughput for 1-week to 1-year commitments, Standard PayGo is default pay-per-use, Priority PayGo adds priority at premium rate, Flex PayGo is discounted with slower response, and batch inference is asynchronous at discounted rate. A 1-year commitment is priced at a 26% discount to 1-month, while Flex PayGo and batch cut costs by about 50% for latency-tolerant work. Spillover above tokens-per-minute flows to Standard PayGo by default or Priority PayGo for more reliability.

Response streaming with streamGenerateContent sends tokens before completion, improving time to first token (TTFT) and time to last token (TTLT). Controls that reduce TTFT and TTLT include smaller models such as Flash-Lite, smaller prompts with common content first, limiting output with max_output_tokens, limiting thinking budget, and using Provisioned Throughput. Elastic compute absorbs spikes signaled by 429 resource-exhausted errors through GKE autoscaling, Cloud Run scaling between minimum and maximum instances to zero when idle, exponential-backoff retries, hybrid Provisioned Throughput plus PayGo, global endpoint routing, and shifting deferrable work off-peak.

Key Facts

• BigQuery ML trains with SQL where data lives and requires data in BigQuery, SQL builders, and a supported model type, while AutoML for text, video, or tabular models requires tolerance for greater than 100ms latency.
• The TRANSFORM clause defines preprocessing that is automatically reused at prediction time, ML analytic functions require an empty OVER() clause, and boosted trees and DNNs require EXPORT MODEL to Cloud Storage for inspection.
• A remote model invokes a Vertex AI model through a Cloud Resource connection with the Vertex AI User role, where ML.GENERATE_TEXT returns language, ML.GENERATE_EMBEDDING returns vectors, and row failures return NULL with a default threshold of 1.0.
• The vLLM container supports only text-generation and image-text-to-text models, Vertex AI online prediction needs a minimum of two nodes for high availability, and RAG Engine corpora backed by Vector Search 2.0 are available only in us-central1 with only RagManagedDb supporting CMEK.
• Supervised Gemini tuning freezes base parameters and learns only adapter weights with limits of 131,072 tokens per example and 1 GB JSONL, while Provisioned Throughput reserves guaranteed throughput with spillover to PayGo and Flex PayGo and batch cut costs by about 50%.