Choosing ARIMA, DNN, and LLM Models

Analyzing data characteristics and problem formulation means matching the structure, volume, and timing of data to the model that can learn from it and serve it. Ordered timestamped numbers point to forecasting, large high-dimensional data with complex patterns points to deep neural networks, and text or combined media that must be understood or created points to large language and multimodal models. The final choice on Google Cloud is then limited by modality fit, response quality, latency, traffic, and cost.

1.1Time-Series Forecasting with ARIMA_PLUS in BigQuery ML

BigQuery ML ARIMA_PLUS is the model type for univariate time-series forecasting where each row pairs a time point with a numeric value to predict. The problem requires a time_series_timestamp_col that supplies ordered time points and a time_series_data_col that supplies the numeric values to forecast, both selected from a BigQuery table. This structure learns trend, seasonality, and dependence over time, so it fits future sales and other ordered numeric forecasts rather than fixed-feature classification or language generation.

Training and forecasting with ARIMA_PLUS follows an ordered SQL workflow:

1. Create model: Run CREATE OR REPLACE MODEL with OPTIONS(model_type = 'ARIMA_PLUS', time_series_timestamp_col, time_series_data_col, auto_arima = TRUE, data_frequency = 'AUTO_FREQUENCY', decompose_time_series = TRUE) over a SELECT of timestamp and value columns.
2. Generate forecast: Run SELECT * FROM ML.FORECAST(MODEL, STRUCT(HORIZON AS horizon, CONFIDENCE_LEVEL AS confidence_level)) where HORIZON is an INT64 count of future time points and CONFIDENCE_LEVEL is a FLOAT64 percentage of future values expected inside the prediction interval.
3. Serve insights: Read forecasted values for future dates from the query results and pass them to downstream applications for decisions.

Setting auto_arima = TRUE lets BigQuery ML (opens in a new tab) search autoregressive and moving-average structure automatically, data_frequency = 'AUTO_FREQUENCY' infers time granularity, and decompose_time_series = TRUE separates components for interpretation. Because output depends directly on the timestamp column and horizon, missing timestamps, irregular frequency, or inaccurate inputs cause weak performance or false outputs.

1.2Deep Neural Architectures for Complex Pattern Recognition

Deep learning foundation models are large models trained on multi-terabyte datasets of text, images, code, and other media to find complex patterns and build contextual understanding. Data volume plus model architecture lets the model produce new content such as text, images, music, or video shaped by its training data. This capability fits problems where inputs are high-volume, high-dimensional, and non-temporal, and where relationships are too complex for explicit time-series equations.

The dominant architecture for these models is the Transformer, developed by Google in 2017, which underlies large-scale models trained on billions of text samples and other content. Foundation models built on Transformers show emergent abilities where one text prompt lets the same model translate languages, answer questions, write a poem, or write code without separate training for each task. A model can then adapt to a domain with prompt techniques or fine-tuning on minimal additional data, so data volume shifts from training from scratch to supplying enough representative examples for adaptation.

1.3Generative Language and Multimodal Reasoning with Large Language Models

Large language models (LLMs) are foundation models trained on text that specialize in translation, question answering, summarization, and code generation. An LLM is the right choice when the task requires understanding context, following instructions, or generating fluent new text rather than forecasting a numeric series or classifying fixed features. LLMs can be customized for specific domains where vocabulary, style, and factual grounding matter.

Multimodal models extend LLM ability beyond text to joint reasoning over multiple input types:

• text
• images
• video
• audio
• code

A multimodal prompt combines formats, for example sending an image and asking the application to list or describe objects in the image. Gemini models are built for multimodality and reason across text, images, video, audio, and code in one model. Vertex AI (opens in a new tab) connects discovery to serving, so the serving path flows from model discovery in Model Garden to customization and deployment in Vertex AI.

1.4Matching Modality, Cost, and Latency to Model Choice

Model selection balances modality fit, size, cost, response quality, latency, traffic, and physical resources. The documented rule is to choose the most affordable model that still meets response quality and latency requirements. Modality must align with the use case, so text-to-image generation depends on a model trained on both text and image data.

Operational demands further separate lightweight forecasting from large-model serving. ARIMA_PLUS training and ML.FORECAST run as SQL in BigQuery, while LLM inference often needs GPUs and tensor processing units (TPUs) and serving frameworks such as vLLM on single-host GPU nodes in Google Kubernetes Engine (GKE) (opens in a new tab). After choosing a deployment environment, anticipated traffic, latency needs, and budget must be balanced against physical resources, where lower cost can mean accepting higher latency on lower-cost machines in test environments.

Integrating Google Cloud managed services into model selection means choosing the model type together with the platform that will train, serve, and operate it. Vertex AI (opens in a new tab) provides managed training, pre-trained foundation models, Vertex AI AutoML for code-free training, managed endpoints for serving, and registry, evaluation, and monitoring for lifecycle management. The correct choice is the most affordable model that still meets response quality and latency, because cost, operational overhead, and scalability differ sharply between a pre-trained LLM, an AutoML model, and a custom deep neural network (DNN).

2.1Foundation Model Discovery and Choice on Vertex AI

Foundation models on Vertex AI are large models trained on multi-terabyte datasets of text, images, code, and other media that can find complex patterns and generate new content. Foundation models provide emergent abilities where a simple text prompt performs translation, question answering, and code writing without explicit training for each task, and they adapt through prompt design or fine-tuning on minimal additional data. Large language models are one type of foundation model based on deep-learning architectures such as the Transformer, trained on billions of text samples and customizable for specific domains.

Model Garden is the catalog for finding and customizing foundation models from Google, open-source, and third-party sources for use on Vertex AI. Multimodal models in Model Garden extend applications beyond text to images, video, audio, and code, and Gemini models reason across all of those modalities in a single model. Gemini 2.5 Flash is designed for agentic workloads with higher speed, built-in tool use, multimodal generation, and a 1M token context window. Choose a foundation model or LLM when broad language or multimodal understanding and development speed matter more than training a task-specific DNN from scratch, and match modality so text-to-image uses a model trained on text and image data.

2.2Efficient Custom Training with Vertex AI AutoML

Vertex AI AutoML is the managed service for training a custom model from a labeled dataset without writing training code, which shortens development when expertise or training infrastructure is limited. Training flows from a Vertex AI dataset into the training service, then the training pipeline trains and validates the model, then the completed artifact is stored for deployment to a managed endpoint. In the console the flow is Datasets page, select dataset, Train new model, select AutoML, choose host location, name the model, optionally set the data split, and Start Training.

AutoML image object detection supports distinct cloud-hosted model types that trade prediction quality against serving latency:

The API exposes the same choice through the training pipeline, with fractionSplit values for training, validation, and test fractions that must sum to 1, for example 0.7, 0.15, and 0.15. Choose AutoML when tabular, image, text, or video data fits a supported task and faster iteration outweighs full architectural control, and choose a custom DNN or open LLM when the task needs novel architectures, hardware control, or exportability that AutoML cloud models do not provide.

2.3Deployment Paths on Managed Endpoints and Self-Managed GKE

The deployment path directly constrains model selection because managed and self-managed environments own different parts of scaling, hardware, and operations. Vertex AI Endpoints is the managed serving platform where inference runs on Google-managed infrastructure, which reduces the need for the team to provision GPU or TPU quota directly. Google Kubernetes Engine (GKE) (opens in a new tab) is the self-managed option where the team controls clusters, node pools, and serving frameworks but also manages scaling, upgrades, and resources, including serving open models such as Gemma, Llama 4 Scout, Qwen3, and gpt-oss with the vLLM framework.

Hardware selection creates an explicit cost and latency tradeoff before committing to a large model. Training node pools can use A100, H100, H200, or B200 GPUs for throughput, while L4 GPUs are commonly used for inference, and lower-cost machines can be tolerated when higher latency is acceptable in test environments. Infrastructure as Code with Terraform makes either path repeatable: define GKE clusters with optimized node pools, Agent Platform endpoints with machine types and scaling policies, and Cloud Storage buckets for data and artifacts, then organize the configuration as modular templates and integrate changes into continuous integration and continuous delivery pipelines with Cloud Build. Choose managed Vertex AI endpoints when operational simplicity and autoscaling matter most, and choose GKE self-hosting when fine-grained control over GPUs, networking, co-located services, or custom serving stacks is required.

2.4Standardized Evaluation from Prototype to Production

Standardized evaluation on Vertex AI Model Evaluation lets teams compare ARIMA, DNN, AutoML, and LLM options on the same criteria before promoting a prototype to production. Computation-based metrics are calculated by numeric formulas and may differ from training loss, so a fraud model trained with cross-entropy loss is better judged by recall, precision, and area under the precision-recall curve (AuPRC). Qualitative traits such as fluency or entertainment value cannot be computed objectively and require LLM-as-a-judge evaluation or human labeling.

A reproducible evaluation pipeline follows an ordered sequence:

1. Define indicators: Set quantitative thresholds such as minimum accuracy, maximum error rate, and minimum F1-score, plus generative checks for factual accuracy, safety, instruction adherence, grounding, style, and verbosity.
2. Choose data and techniques: Use a golden dataset that reflects production distribution for statistical significance, LLM-as-a-judge only where an LLM can validly judge the task, and direct user feedback from production traffic, with BigQuery for SQL-accessible inference data and Cloud Storage for cost-efficient large datasets.
3. Standardize the pipeline: Automate training and evaluation jobs, version datasets and code, and log parameters and results with Experiments so iterations are comparable and auditable.

Vertex AI rapid evaluation runs automatic evaluations from a provided dataset and prompts during development, while adversarial testing, subgroup analysis, and counterfactual generation expose vulnerabilities and bias before deployment. Insights from development evaluation then define the production monitoring strategy, so evaluation thresholds become monitoring alerts and retraining triggers under operational excellence (opens in a new tab) practices.

2.5MLOps Lifecycle with Registry, Monitoring, and Observability

Sustainable lifecycle management connects Vertex AI Model Registry, Vertex AI Model Monitoring, and Vertex AI Pipelines into a feedback loop from serving back to retraining. Model Registry stores versioned artifacts and lineage after each successful training run, so a rollback can revert traffic to the most recent stable version when quality degrades. Model Monitoring v2 tracks input feature drift, output drift, and feature attribution for models on managed endpoints and self-hosted GKE services, and it can monitor at model level for granular tracking rather than only at endpoint level.

Training-serving skew detection depends on a golden dataset and schema baseline: analyze training data with TensorFlow Data Validation (TFDV) to establish the baseline schema, then configure Model Monitoring to compare live input distribution against that baseline and alert on skew or prediction drift. Traditional models are monitored with accuracy, precision, recall, F1-score, AUC-ROC, and log loss, while generative AI uses the Gen AI evaluation service to monitor response quality, safety, grounding, and verbosity, with human review for subjective quality. When an alert fires, the pipeline can automatically retrain with Vertex AI custom training, including hyperparameter tuning and distributed training, or re-run fine-tuning for foundation models.

Availability and reliability observability sits alongside model-quality monitoring and uses the same operational stack. Cloud Monitoring collects request rate, error rate, latency, and GPU or TPU utilization, including LLM-specific time to first token (TTFT) in Metrics Explorer and throughput and error rates in the model observability dashboard. Cloud Logging provides event and error context, Cloud Trace follows request flows across distributed inference microservices, and Cloud Profiler finds CPU and memory bottlenecks in application code. Reliability goals are expressed as service-level objectives tied to business metrics, such as 99.9% successful API responses, 95th percentile latency below 300 ms, TTFT below 500 ms for 99% of requests, and harmful output below 0.1%, with the four golden signals of latency, traffic, error rate, and saturation guiding scaling under reliability (opens in a new tab) practices.

Evaluating operational constraints means comparing model complexity against production needs for latency, throughput, resource use, and scalability on Google Cloud Platform (GCP) infrastructure. A larger, more capable model increases accelerator, memory, and storage demands while raising inference latency and cost, so production selection balances response quality against speed, availability, and budget. On Google Kubernetes Engine (GKE) (opens in a new tab), that balance is enforced through latency and throughput metrics, weight-loading storage choices, accelerator provisioning, and traffic controls.

3.1Defining Inference Latency for Production Model Decisions

Inference latency metrics quantify how quickly a deployed model, especially an LLM, responds to a request on GKE. Request latency is the end-to-end time to complete a request, but LLM serving splits that time into prefill and decode phases that affect user experience differently. Choosing a model type requires matching these components to service-level objectives, such as 95th percentile inference latency below 300 ms or time to first token below 500 ms for 99% of requests.

• Time to First Token (TTFT) (ms): Time from request arrival to generation of the first output token; TTFT captures prefill and scheduling delay for interactive chat and agents.
• Time Per Output Token (TPOT) (ms): Average decode speed measured as (request_latency - time_to_first_token) / (total_output_tokens - 1); TPOT captures streaming smoothness after the first token.
• Inter-token latency (ITL) (ms): Time between two individual output token generations, aggregated as mean, median, and percentiles such as p90; unlike TPOT, ITL exposes stalls within a single request.
• Normalized Time Per Output Token (NTPOT) (ms): Request latency normalized by output length, measured as request_latency / total_output_tokens; NTPOT allows comparison across requests with different output lengths.
• Request latency (ms): End-to-end time to complete a request, combining queueing, prefill, decode, and post-processing on the serving stack.

3.2Measuring Throughput for Generative Inference Workloads

Throughput metrics quantify how much work an inference platform completes per second and determine how many accelerators and replicas a model family requires. Throughput on GKE depends on input and output context lengths, batching in serving frameworks such as vLLM and NVIDIA Triton Inference Server, and accelerator saturation, so raw request counts alone can mislead capacity planning. Production monitoring collects these metrics through Cloud Monitoring and Google Cloud Managed Service for Prometheus, with the model observability dashboard reporting queries per second (QPS), token throughput, and first-token latencies.

• Requests per second: Total requests served per second; this metric varies widely with context length and is not reliable for LLM throughput comparison.
• Output tokens per second: Measured as total_output_tokens_generated_by_server / elapsed_time_in_seconds; this metric reflects decode capacity for generative workloads.
• Input tokens per second: Measured as total_input_tokens_generated_by_server / elapsed_time_in_seconds; this metric reflects prefill load from long prompts.
• Tokens per second: Measured as total_tokens_generated_by_server / elapsed_time_in_seconds, counting both input and output tokens; this metric helps compare workloads with high prefill versus high decode times.

3.3Selecting Models by Modality, Size, and Cost

Model selection on GCP starts from modality, size, and cost, with the rule to choose the most affordable model that still meets response quality and latency requirements. Modality is the high-level data category a model is trained for, such as text, images, video, audio, and code, and the use case must match the modality; text-to-image generation requires a model trained on text and image data. Size drives complexity and resource consumption: foundation models and LLMs based on Transformer architectures train on billions of samples and need GPUs or TPUs for serving, while smaller models tolerate lower-cost machines at higher latency in test environments.

3.4Reducing Weight-Loading Delay with Storage Choice

Weight-loading latency is the startup delay when inference Pods load large model weights from storage, and the storage choice controls scale-out consistency on GKE. Data flows from the storage system to each inference Pod during startup and scaling events, so the storage service depends on Pod count and throughput requirements. The deployment depends on matching limited-scale simplicity against parallel high-throughput access for thousands of replicas.

3.5Provisioning Accelerators for Availability and Cost Control

Accelerator provisioning trades cost-efficiency against guaranteed capacity for training and inference on GKE and Compute Engine. Training typically uses A100, H100, H200, or B200 GPUs, while inference often uses L4 GPUs, and the vLLM (opens in a new tab) engine can dynamically use TPUs and GPUs to address fluctuating capacity and performance needs. Availability planning must also account for quotas, historical utilization in Cloud Monitoring, load testing, and proactive quota increases through the Google Cloud console.

• Committed use discounts (CUDs): Discounted pricing for virtual machines (VMs) and accelerators with stable 24x7 inference or fixed training capacity; CUDs lower unit cost when demand is predictable.
• Spot VMs: Preemptible capacity that reduces GPU cost for fault-tolerant evaluation and experimentation; Spot VMs can be disrupted and need fallback handling.
• Flex-start with Dynamic Workload Scheduler (DWS): Short-term reservations up to seven days with pay-as-you-go billing and discounts up to 53% for vCPUs, GPUs, TPUs, and attached Local SSD disks; GKE persists the GPU request and provisions Flex-start VMs when capacity becomes available.
• DWS Calendar mode: Reservations up to three months for specific H100 and H200 GPUs and Trillium TPUs; Calendar mode suits longer-running workloads that need obtainability guarantees.
• GKE custom compute classes: Hierarchical node configuration profiles with fallback policies for autoscaling; a profile can prioritize reserved GPU or TPU instances and fall back to Spot VMs when reserved resources are unavailable.

3.6Distributing Traffic and Meeting Reliability Targets

Traffic distribution spreads incoming inference load across multiple instances to optimize utilization, provide high availability, and preserve low latency under fluctuating demand. Traffic flows from the load balancer to model servers behind health checks, and GKE Inference Gateway adds model-aware routing that prioritizes instances with GPU and TPU accelerators for generative AI and LLM inference. For diverse platforms, a centralized balancer can route low-GPU requests to Cloud Run and GPU-intensive tasks to GKE, while global users use a global external Application Load Balancer for low-latency distribution.

Reliability goals translate business key performance indicators into technical objectives measured across latency, traffic, error rate, and saturation of CPU, memory, GPU, and TPU resources. Example targets are 99.9% of API calls returning success, 95th percentile inference latency below 300 ms, TTFT below 500 ms for 99% of requests, and harmful output below 0.1%. After deployment, continuous monitoring tracks request rate, error rate, latency, resource utilization, and generative output quality, with alerts triggering rollback to the prior stable version tracked in Model Registry under reliability (opens in a new tab) practices.

Key Facts

• BigQuery ML ARIMA_PLUS forecasts univariate time series from a timestamp column and numeric value column using HORIZON as INT64 future points and CONFIDENCE_LEVEL as FLOAT64 prediction-interval percentage.
• Vertex AI AutoML trains custom models without training code, where image object detection offers non-exportable CLOUD_1, CLOUD_HIGH_ACCURACY_1, and CLOUD_LOW_LATENCY_1 with different quality-latency tradeoffs.
• LLM inference latency splits into Time to First Token for prefill delay and Time Per Output Token for decode speed, while LLM throughput should be compared with output tokens per second or total tokens per second, not requests per second.
• Large-model scale-out on GKE depends on weight-loading storage, where Cloud Storage FUSE suits limited scaling, Google Cloud Managed Lustre attaches to 10,000 or more Pods, and Hyperdisk ML attaches to up to 2,500 Pods.
• Production reliability is enforced with objectives such as 99.9% successful responses, 95th percentile latency below 300 ms, and rollback to the stable version in Model Registry when monitoring detects drift or quality loss.