Prepare and test your skills

Prepare and test your skills

Analysis of Cloud TPUs (opens in a new tab) shows how specialized accelerators run machine learning (ML) workloads faster than general-purpose processors. Specialized accelerators are hardware built for one type of math-heavy work, while general-purpose processors handle many different tasks. On Google Cloud, Compute Engine offers graphics processing unit (GPU) machine types that accelerate ML, data processing, and graphics-intensive workloads.
Accelerator-optimized machine series are Compute Engine families that bundle GPUs with virtual machines (VMs) for specific workloads. The A series is designed for high-performance computing (HPC), artificial intelligence (AI), and ML workloads. Later-generation A series machine types are ideal for pre-training and fine-tuning foundation models on large clusters, while the A2 series trains smaller models and runs single-host inference. The G series is designed for NVIDIA Omniverse simulation, graphics-intensive applications, video transcoding, and virtual desktops, and it supports NVIDIA RTX Virtual Workstations (vWS).
Accelerator-optimized A and G series attach the GPU to the instance automatically. N1 general-purpose instances work differently because the user attaches a select GPU model, such as NVIDIA T4, P4, and V100. N1 attachment is not allowed on the shared-core f1-micro and g1-small types. The G series can also train smaller models and run single-host inference when full HPC power is not needed.
GPU architectures are hardware generations that determine which numeric precisions their Tensor Cores can accelerate. The Blackwell architecture is used by the A4X Max, A4X, A4, and G4 machine types and adds Tensor Core support for FP4 precision and expanded INT4 capabilities for large-model inference. The Hopper architecture powers the A3 series, the Ampere architecture powers the A2 series, and the Ada Lovelace architecture powers the G2 series.
Precision formats control the tradeoff between speed and numerical accuracy. Lower-precision formats such as FP4 and INT8 process more calculations per second, which speeds up inference, while higher-precision formats such as FP64 support demanding HPC and scientific workloads. Other supported formats include TF32, mixed FP16/32, and FP8. Structural sparsity can double computational throughput for TF32, FP16/32, INT8, FP8, and FP4, and Blackwell-based GPUs also support the bfloat16 data type for mixed-precision training.
Matching an accelerator to a workload means comparing its memory, architecture, and precision support to what the job needs. The G2 series uses the Ada Lovelace architecture with 24 GB of GDDR6 memory, and its NVIDIA L4 GPUs provide up to four times the inference performance of NVIDIA T4 GPUs. The G4 series uses the Blackwell architecture with 96 GB of GPU memory per GPU.
| Option | Best for | Architecture and memory |
|---|---|---|
| G2 with NVIDIA L4 | High-performance AI inference, generative AI, video processing and streaming, graphics, and light ML training | Ada Lovelace with 24 GB GDDR6 |
| G4 with NVIDIA RTX PRO 6000 | High-performance simulation, NVIDIA Omniverse, 3D graphics rendering, local large language model (LLM) fine-tuning and inference, virtual workstations, and fractional GPU sharing | Blackwell with 96 GB per GPU |
Exam tip: The G2 series is tuned for AI inference and light training, while the G4 series is tuned for simulation, rendering, and local LLM fine-tuning.
AI Hypercomputer (opens in a new tab) is a supercomputing system optimized for AI and ML workloads. AI Hypercomputer is recommended for creating densely allocated, performance-optimized infrastructure with integrations for Google Kubernetes Engine (GKE) (opens in a new tab) and Slurm schedulers. Some GPU machine types can run on AI Hypercomputer when the workload needs tightly connected accelerators.
The A4X Max and A4X machine series run on an exascale platform based on NVIDIA's rack-scale architecture. This platform is optimized for compute- and memory-intensive, network-bound ML training and HPC workloads. A4X Max uses NVIDIA GB300 Grace Blackwell Ultra Superchips and is available only as bare metal instances, which give direct access to the host server's CPU and memory without the Compute Engine hypervisor layer.
Explanation of TensorFlow shows it is an end-to-end, open source framework for building and training ML models. Google open sourced TensorFlow in 2015, and it has become one of the most popular ML frameworks. Its purpose is to support the full model lifecycle in one platform, from developing a model to training it on large datasets and deploying it for use.
What TensorFlow provides across environments is a common set of tools for developing, training, and deploying models on many different systems. Data is at the heart of this work because training a deep learning model requires collecting and storing a lot of data. The speed of moving that data from its storage location to the training process is increasingly important. That movement matters more as accelerators such as GPUs and Cloud TPUs make the training math itself faster, so slow data delivery can become the bottleneck.
TensorFlow Enterprise is Google Cloud's enterprise version of TensorFlow for AI-enabled businesses on Google Cloud. TensorFlow Enterprise (opens in a new tab) is designed for users who deploy TensorFlow on Google Cloud and want enterprise-grade support and performance from the creators of TensorFlow. TensorFlow Enterprise makes accessing data on Google Cloud faster and easier. That faster path from cloud storage to the training process keeps data flowing steadily to the accelerators.
How Cloud TPUs (opens in a new tab) accelerate TensorFlow workloads is by taking over heavy math operations from general-purpose compute. Tensor Processing Units (TPUs) are Google's custom-developed, application-specific integrated circuits designed to accelerate ML and AI workloads. A TensorFlow training job sends its heavy math operations to the TPU, and the TPU returns the results much faster than general-purpose compute could. TensorFlow workflows using TPUs can run on Compute Engine, which creates and manages individual TPU VMs, and on Google Kubernetes Engine (GKE) (opens in a new tab), which provides a fully managed Kubernetes environment for large-scale training and inference.
Cloud TPUs achieve this acceleration through three related capabilities:
Because of this design, Cloud TPUs fit TensorFlow models that are dominated by matrix computations, use large effective batch sizes, and train for a long time. Examples include large models and ultra-large embeddings for ranking and recommendation.
Exam tip: TensorFlow is open source software for building and training models, while Cloud TPU is Google's proprietary hardware that accelerates those models.
Evaluation of TensorFlow and Cloud TPUs shows they form a combined software and hardware path for ML on Google Cloud. TensorFlow provides the end-to-end open source tools for building and training models, while Cloud TPUs provide the specialized infrastructure that runs those models with high performance. Google Cloud manages where TPUs are available, how they are configured, and how they scale through Compute Engine and Google Kubernetes Engine.
The TensorFlow toolkit from preparation to deployment covers the full model workflow from data preparation to training to deployment. Google open sourced TensorFlow in 2015 as an end-to-end open source framework for building and training ML models. TensorFlow Enterprise (opens in a new tab) is the Google Cloud version for businesses that need enterprise-grade support and performance from the creators of TensorFlow. TensorFlow Enterprise focuses on making data access faster and easier, because training speed depends on how quickly data flows from its storage location to the training process running on accelerators such as GPUs and Cloud TPUs.
Cloud TPU hardware for training and inference at scale consists of Google's custom application-specific integrated circuits (ASICs) for ML and AI. Cloud Tensor Processing Units (Cloud TPUs) (opens in a new tab) are engineered for both long training jobs that run for weeks and large-scale inference. Three hardware features drive their performance for TensorFlow workloads. Matrix Multiply Units (MXUs) execute the massive matrix operations at the core of ML algorithms, on-chip high-bandwidth memory (HBM) lets teams train larger models with larger batch sizes, and TPU chips can be joined into groups called slices that scale a single job up to thousands of chips.
Workload patterns that fit TPUs are jobs with steady, large-scale math where training data flows from storage such as Cloud Storage to the TensorFlow training process running on the TPU. Jobs with large matrix computations benefit most from MXUs, HBM, and slices.
A decision tree that checks matrix dominance, custom operations, branching and precision needs, and scale to decide between Cloud TPU and CPU or GPU.
Models well suited for TPUs include:
TPUs are not suited for workloads that need frequent branching, high precision, or custom logic in the training loop:
Provisioning and scaling TPUs with managed services is how Google Cloud controls availability, configuration, and scaling of TPU capacity. Compute Engine creates and manages individual TPU VMs or slices with full lifecycle management. Google Kubernetes Engine (GKE) (opens in a new tab) provides a fully managed, multi-tenant Kubernetes environment that orchestrates large-scale AI workloads and manages the lifecycle of TPU nodes and node pools, including creating, configuring, and deleting TPU VMs.
| Option | What it manages | Choose when |
|---|---|---|
| Compute Engine | Individual TPU VMs or TPU slices | The team needs direct control over the lifecycle of TPU VMs |
| Google Kubernetes Engine (GKE) | TPU nodes and node pools inside Kubernetes clusters | The workload needs Kubernetes orchestration across many AI jobs |
| Cloud TPU API | Legacy TPU provisioning through API, CLI, and client libraries | Only for existing resources, because it receives bug fixes and security updates only |
Compute Engine supports TPU7x (Ironwood), TPU v6e (Trillium), and TPU v5p. A TPU slice lets a TensorFlow job scale from one VM to many connected chips, so the same model code can grow from testing to massive training without changing platforms.
Exam tip: Use Compute Engine or GKE for new TPU resources, because the Cloud TPU API is no longer under development and is not supported on TPU7x and later versions.
Project setup steps for TPU VMs define the project, billing, APIs, permissions, and identity the TPU VMs need to access other resources. The TPU VM depends on a user-managed service account to access services such as storage and logging, and the user depends on Identity and Access Management roles to create and manage those resources. Setting up the project (opens in a new tab) follows a fixed order:
compute.googleapis.com with gcloud or the console so the project can create TPU resources.TensorFlow is an end-to-end open source framework for building and training machine learning models from development to training to deployment, while Cloud Tensor Processing Unit (TPU) is Google's custom application-specific integrated circuit hardware that accelerates those models. A TensorFlow training job sends its heavy math operations to the Cloud TPU, which returns the results much faster than general-purpose compute could.
Cloud Tensor Processing Units (TPUs) fit best for TensorFlow models dominated by large matrix computations, with large effective batch sizes, long training runs lasting weeks or months, and ultra-large embeddings for ranking and recommendation. They use Matrix Multiply Units for matrix operations, high-bandwidth memory for larger models and batch sizes, and slices that scale to thousands of chips. They fit poorly for workloads with frequent branching or many element-wise operations, workloads requiring high-precision arithmetic, or neural networks with custom operations in the main training loop.
Use Compute Engine when the team needs direct control over the lifecycle of individual Tensor Processing Unit virtual machines or slices, since it creates and manages individual TPU virtual machines or slices. Use Google Kubernetes Engine (GKE) when the workload needs Kubernetes orchestration across many artificial intelligence jobs, since it provides a fully managed Kubernetes environment that manages the lifecycle of TPU nodes and node pools.
An organization is using TensorFlow to build and train a large-scale machine learning model. The engineering team needs to provision and manage Cloud Tensor Processing Units (TPUs) using Compute Engine managed instance groups (MIGs) to support the following workload requirements:
Which Cloud TPU configuration should the organization implement to meet these requirements?
Last updated