Prepare and test your skills
Prepare and test your skills
Google's global infrastructure is the planetary-scale foundation of data centers, networking, compute, storage, and software that supports enterprise generative AI. The foundation reflects over two decades of running planet-scale AI in production. AI Hypercomputer (opens in a new tab) sits on top of this foundation as an integrated supercomputing system that co-designs hardware, networking, storage, and software to improve performance across training, tuning, and serving.
Planetary-scale data centers are Google's globally distributed facilities that supply the raw capacity for training, tuning, and serving large generative models. Training flows from development tools to large clusters of accelerators in these data centers, tuning refines the models on the same capacity, and serving delivers results to users from locations close to demand. Because capacity is pooled across many regions, a workload can scale up for a large training run and scale out for high-volume inference without depending on a single site. This pooling supports reliability through redundancy across sites and cost-efficiency through shared, flexible consumption.
High-speed networking is the connective layer that moves data between compute, storage, and users across Google's global infrastructure. Data flows from storage to accelerator clusters for training, between nodes inside a cluster during distributed computation, and from serving systems to end users during inference. The network depends on tight integration with compute and storage, so a bottleneck in one layer does not stall the others. This co-design of networking with compute, storage, and software provides the continuous throughput needed when large datasets and model parameters must move quickly.
Enterprise-grade generative AI needs throughput, reliability, and cost-efficiency at the same time, and Google's global infrastructure delivers all three together:
AI Hypercomputer combines these benefits by bringing performance-optimized hardware, open software, leading frameworks, and flexible consumption into one system. Teams can start from a model in the Model Garden, configure deployment on Google Kubernetes Engine (GKE) (opens in a new tab), and deploy to production on the same optimized stack. The result is higher efficiency because infrastructure, software, and lifecycle stages are managed as one integrated system rather than separate parts.
Exam tip: Global data centers and networking provide scale and movement, while AI Hypercomputer provides the co-design of compute, storage, networking, and software that turns that scale into efficient training, tuning, and serving.
Specialized AI hardware accelerators are performance-optimized chips that perform the heavy math behind generative AI training and serving. AI Hypercomputer (opens in a new tab) brings these accelerators together with high-bandwidth networking, high-throughput storage, open software, and flexible consumption models. The system vertically integrates those layers to maximize goodput, the measure of actual machine learning productivity, so accelerators spend more time on productive computation and less time on system overhead.
Graphics Processing Units (GPUs) are NVIDIA accelerators in AI Hypercomputer that handle a wide range of AI and machine learning workloads, from prototyping to large-scale training and real-time serving. Google Cloud manages GPUs in two different ways depending on whether workloads run independently or must stay tightly synchronized. General GPUs work well for serving fleets made of independent tasks, while clustered GPUs work well for large training jobs where many nodes must act as one system.
| GPU type | Management unit | Maintenance and reliability |
|---|---|---|
| General GPUs | Independent compute units | Asynchronous maintenance with standard node auto-repair to maintain pod-level uptime for serving fleets |
| Clustered GPUs | High-performance cluster managed as a single tightly coupled system | Synchronized maintenance with automated hardware health monitoring and proactive node replacement |
Choose clustered GPUs when large-scale training depends on many nodes acting as one reliable system. Choose general GPUs when a serving fleet needs independent units that can be repaired without stopping the whole system.
Tensor Processing Units (TPUs) are Google's custom-designed accelerators built specifically for the large matrix operations at the heart of generative models. Each TPU chip (opens in a new tab) includes on-chip high-bandwidth memory (HBM) that holds larger models and larger batch sizes during training and inference. TPUs connect in groups called slices that scale a workload with little to no code changes, so added chips increase capacity without rewriting the application. TPU slices take two forms that map to different needs: multiple single-host TPU virtual machines (VMs) run multiple independent workloads, which fits inference fleets where each VM serves its own requests, while a multi-host TPU slice connects multiple interlinked TPU VMs that work together as a unit on a single workload, which fits distributed training where all nodes must cooperate.
Accelerator performance at scale comes from the network, storage, software, and provisioning around the chips. Accelerators connect over high-bandwidth networking to sustain throughput across thousands of compute instances, while high-throughput storage services such as Google Cloud Managed Lustre and Cloud Storage Rapid move data fast enough to eliminate input and output bottlenecks. Optimized versions of PyTorch, JAX, and TensorFlow run efficiently on GPUs and TPUs, and orchestration through Google Kubernetes Engine (GKE), Slurm through Cluster Director, or Cluster Toolkit blueprints places jobs on the right accelerators. Flexible provisioning ensures a large training job gets all of its interconnected nodes together, with Dynamic Workload Scheduler providing atomic provisioning for entire blocks of nodes, while Flex-start VMs, Spot VMs, and Reservations support prototyping, development, and steadier workloads.
Exam tip: Choose single-host TPU virtual machines when you have multiple independent inference workloads, and choose a multi-host TPU slice when multiple interconnected TPU virtual machines must work together as a unit on one workload.
AI Hypercomputer (opens in a new tab) is an integrated supercomputing system that is optimized for artificial intelligence and machine learning workloads. AI Hypercomputer vertically integrates performance-optimized hardware, open software, and flexible consumption models into one unified portfolio. This integration is designed to maximize goodput, the measure of actual machine learning productivity, by helping accelerators spend more time on productive computation and less time on system overhead. It supports the full AI development lifecycle, from prototyping and development to massive-scale training and real-time serving.
Three-layer stack with consumption options on top, open software with GKE orchestration in the middle, and GPUs, TPUs, networking and storage at the base to maximize goodput.
Performance-optimized hardware in AI Hypercomputer provides the accelerators, networking, and storage that complex AI workloads depend on. Accelerators connect over high-bandwidth networking so throughput stays high even when a model runs across thousands of compute instances. High-throughput storage services, such as Google Cloud Managed Lustre and Cloud Storage Rapid, feed data to those accelerators to eliminate input and output bottlenecks. The system offers NVIDIA GPUs for general parallel compute and Cloud TPUs with hardware designed for large matrix operations and on-chip high-bandwidth memory for larger models and batch sizes.
| GPU type | Management unit | Maintenance and reliability |
|---|---|---|
| General GPUs | Independent compute units | Asynchronous maintenance with standard node auto-repair to maintain pod-level uptime for serving fleets |
| Clustered GPUs | Single tightly coupled high-performance cluster | Synchronized maintenance with automated hardware health monitoring and proactive node replacement |
TPUs scale in groups called slices with little to no code changes. Choose clustered GPUs when large-scale training depends on many nodes acting as one reliable system, and choose general GPUs when serving fleets need independent units that can be repaired without stopping the whole system.
Open software in AI Hypercomputer provides optimized versions of machine learning frameworks and orchestration platforms that deploy and manage accelerators. AI Hypercomputer offers optimized versions of PyTorch, JAX, and TensorFlow, so code runs efficiently on GPUs and TPUs without custom tuning. The orchestration layer then places that code on the right accelerators, scales containers or cluster nodes up and down, and keeps distributed training and serving running. Data flows from the framework job to GKE pods, to a Slurm queue, or to a Cluster Toolkit-built cluster, and then to the underlying GPUs or TPU slices.
Flexible consumption models in AI Hypercomputer provide different ways to provision accelerator capacity based on workload needs. Flexible provisioning lets a team match cost and availability to the job, from short experiments to reserved production capacity to very large training runs that need many interconnected nodes at once. This flexibility prevents large jobs from waiting for capacity or failing because only part of the requested nodes became available.
The Dynamic Workload Scheduler depends on the underlying GPU and TPU clusters because it acquires the whole connected block together before training starts, so a distributed job only launches after all of its nodes are ready.
Exam tip: Clustered GPUs use synchronized maintenance and proactive node replacement as one system, while General GPUs use asynchronous maintenance and standard auto-repair as independent units.
AI Hypercomputer is an integrated supercomputing system that sits on top of Google's global infrastructure and co-designs hardware, networking, storage, and software to improve performance across training, tuning, and serving. It vertically integrates performance-optimized hardware, open software, and flexible consumption models to maximize goodput, the measure of actual machine learning productivity, so accelerators spend more time on productive computation and less time on system overhead. It supports the full AI development lifecycle from prototyping and development to massive-scale training and real-time serving.
General Graphics Processing Units (GPUs) are managed as independent compute units with asynchronous maintenance and standard node auto-repair to maintain pod-level uptime for serving fleets, while clustered Graphics Processing Units (GPUs) are managed as a single tightly coupled system with synchronized maintenance, automated hardware health monitoring, and proactive node replacement. Choose clustered GPUs when large-scale training depends on many nodes acting as one reliable system, and choose general GPUs when a serving fleet needs independent units that can be repaired without stopping the whole system.
Tensor Processing Units (TPUs) are Google's custom-designed accelerators built specifically for the large matrix operations at the heart of generative models, with on-chip high-bandwidth memory that holds larger models and larger batch sizes and scaling in slices with little to no code changes. Use multiple single-host TPU virtual machines for multiple independent workloads such as inference fleets where each virtual machine serves its own requests, and use a multi-host TPU slice that connects multiple interlinked TPU virtual machines working together as a unit for distributed training where all nodes must cooperate.
Dynamic Workload Scheduler provides atomic provisioning of entire blocks of interconnected nodes so a large-scale training job launches only after all nodes are ready. It depends on the underlying Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs) clusters because it acquires the whole connected block together before training starts, which prevents large jobs from waiting for capacity or failing because only part of the requested nodes became available.
An enterprise is planning to train and serve large-scale generative AI foundation models and is evaluating Google Cloud's AI-optimized infrastructure to support these intensive workloads.
Which statement best describes the architecture and core components of Google Cloud's AI Hypercomputer?
An integrated system that combines performance-optimized hardware, open software frameworks, and flexible consumption models to efficiently scale AI workloads.
A serverless web application hosting runtime designed exclusively for converting Python scripts into microservices without physical compute resources.
A standalone local desktop workstation equipped with client-side tensor processing hardware intended strictly for offline experimentation.
A single monolithic mainframe server residing in one data center that relies entirely on proprietary code and rigid multi-year hardware leases.
Last updated