Design, implement, and secure Google Cloud networks, from VPCs and hybrid connectivity to load balancing and Cloud Armor, and track your readiness for the Professional Cloud Network Engineer exam.
Prepare and test your skills
Prepare and test your skills
Optimizing backend traffic distribution, session affinity, and connection lifecycles ensures client requests are handled reliably across compute resources without dropping active sessions. Google Cloud uses Cloud Load Balancing, Cloud Service Mesh, and Google Kubernetes Engine (GKE) to spread load across backend types like Managed Instance Groups (MIGs) and Network Endpoint Groups (NEGs). Administrators configure policies for traffic splitting, locality-based balancing, capacity limits, and health checks to enable seamless failover and graceful termination of instances.
Locality load balancing policies determine how a load balancer distributes incoming client requests across available endpoints within a specific backend group or zone. In GKE, a resource called GCPBackendPolicy provides declarative control over load balancing modes and backend selection. Traffic can be routed using standard algorithms like round robin, least connections, random, or more advanced methods like locality and consistent hash.
For advanced workloads, Application Load Balancers can use real-time utilization metrics from backends to make smarter routing decisions. Backends send signals about their current load—such as CPU or memory usage—inside HTTP response headers using the Open Request Cost Aggregation (ORCA) standard. The load balancer then uses these ORCA signals to execute weighted policies, ensuring traffic is steered away from overloaded instances and toward healthier ones.
Capacity management policies control how much traffic a backend service accepts before redirecting excess requests or triggering autoscaling. A backend service defines a capacity threshold, which is the maximum requests per second or resource utilization it can handle. When client demand exceeds this local capacity, the load balancer redirects the overflow traffic to the next closest healthy cluster or backend group in another zone.
Autoscaling systems integrate directly with these traffic metrics to dynamically add or remove compute resources. Workloads can be configured to autoscale based on HTTP requests per second or custom metrics, scaling up before performance degrades. Administrators can also configure explicit weight-based traffic splitting to divide traffic percentages across different backend versions or services.
Connection lifecycle management coordinates backend health checks, connection draining, and instance termination to prevent dropped connections during updates or failures. When a backend instance begins to fail its health checks or is being terminated, the load balancer enters a draining state for that endpoint. Draining allows in-flight requests to complete but stops sending new requests to that specific instance, immediately redirecting new traffic to other healthy backends.
Health checks continuously verify the operational state of backend instances using protocols like HTTP/1.1 and HTTP/2. Proper configuration requires aligning client protocols, proxy timeouts, and backend draining settings. This ensures zero downtime during instance autoscaling or unexpected faults. In multi-zonal deployments, precise firewall rules must allow health check probes from Google Cloud's IP ranges to reach all backend endpoints.
Multi-tier gateway architectures separate edge ingress management from internal routing policies by chaining managed load balancers with service mesh proxies. In a simple, single-level pattern, clients send requests directly to a Google Cloud-managed load balancer, which applies security policies and routes traffic to backend services.
In a more advanced two-level pattern, the managed load balancer forwards traffic to a secondary tier of Cloud Service Mesh edge proxies running on MIGs or GKE workloads. This two-level pattern provides operational separation: the first-level load balancer handles tasks like TLS termination and perimeter security, while the second-level edge proxy executes complex routing rules, such as matching HTTP headers. If incoming requests use protocols unsupported by the proxy layer, the traffic is dropped to protect the downstream services.
Managed Instance Groups (MIGs) serve as scalable backend populations for Google Cloud load balancers, distributing traffic across multiple, identical Virtual Machine (VM) instances that the platform automatically manages. A MIG is created from an instance template, and its controller maintains the desired number of instances, replacing any that become unhealthy. You can configure a MIG as either regional (spanning multiple zones for high availability) or zonal (confined to a single zone).
A backend service defines how a load balancer distributes traffic, and you attach MIGs or Network Endpoint Groups (NEGs) to it. The backend service configuration includes balancing modes that control distribution: rate mode distributes based on requests per second, utilization mode distributes based on backend capacity, and connection limits control the maximum concurrent connections per instance. When using NEGs, traffic can be sent directly to individual endpoints like container pods, which is useful for container-native load balancing in GKE environments.
Autoscaling policies determine when a MIG adds or removes instances based on workload demand. You can configure autoscaling using multiple signals monitored through Cloud Monitoring. CPU utilization-based autoscaling triggers scale-out when the average CPU usage across the group exceeds a set threshold. Load balancing serving capacity-based autoscaling uses the backend service's configured capacity limits to trigger scaling. For more precise control, custom metrics-based autoscaling allows scaling based on application-specific signals like request queue length.
Cool-down periods prevent rapid instance flapping by waiting a specified time after a scaling action before evaluating the need for another action, protecting against churn during brief traffic spikes. Scale-in controls determine how aggressively the MIG reduces capacity when traffic decreases. You also configure minimum and maximum instance limits to ensure sufficient capacity while controlling costs. The autoscaling algorithm uses a stabilization window to prevent rapid scale-down that could disrupt service, and these parameters can be adjusted based on your application's startup time and traffic patterns.
Network Endpoint Groups (NEGs) are logical groupings of endpoints—defined by an IP address and port—that serve as backends for Google Cloud load balancers. Specialized NEG types allow you to integrate endpoints from diverse sources, including GKE containers, serverless platforms like Cloud Run, and external on-premises systems, into a unified load balancing setup.
A hybrid connectivity NEG is used to load balance traffic to endpoints outside Google Cloud, such as on-premises data centers or other public clouds, reachable via Cloud VPN or Cloud Interconnect. You create this NEG by specifying the endpoint type as NON_GCP_PRIVATE_IP_PORT and adding the external IP:Port combinations. You must choose a Google Cloud zone geographically close to your external environment to optimize latency. This NEG type can only be used with specific load balancers, and its balancing mode must be RATE for Application Load Balancers or CONNECTION for proxy Network Load Balancers.
You can configure a single backend service to reference a mixture of Google Cloud-based backends and external backends, but only in a specific combination. A backend service can include both zonal NEGs (for Google Cloud VMs or containers) and hybrid connectivity NEGs (for on-premises endpoints). No other mix of backend types is allowed. For GKE, this mixed-backend support is only available with standalone NEGs; Cloud Service Mesh does not support it. To configure this, you either create separate backend services for each type or use the single, mixed-backend service approach, ensuring the backend service protocol matches the load balancer type.
Zonal NEGs group endpoints within a single zone. The GCE_VM_IP_PORT type is primarily used for container-native load balancing in GKE, directing traffic directly to Pod IP addresses, which reduces latency and provides Pod-level visibility. GKE can manage these NEGs automatically, or you can create standalone NEGs for manual configuration.
The GCE_VM_IP zonal NEG type is used exclusively with internal and external passthrough Network Load Balancers. It enables flexible endpoint grouping, allowing a single VM to belong to multiple NEGs, and supports load balancing to non-primary network interfaces, which is useful for third-party appliances. In GKE, GCE_VM_IP NEGs enable subsetting for internal passthrough Network Load Balancers, improving scalability.
Health check configuration depends on the load balancer type. For NON_GCP_PRIVATE_IP_PORT endpoints used with global external Application Load Balancers, centralized health checks are required. This necessitates creating firewall rules on your external network to allow ingress traffic from Google's health check probe IP ranges (35.191.0.0/16 and 130.211.0.0/22).
For Envoy-based load balancers (like regional Application Load Balancers) using hybrid NEGs, distributed Envoy health checks are used. This requires a firewall rule to allow traffic from the region's proxy-only subnet to reach the external endpoints. With distributed health checks, every endpoint attached to a backend service must be unique; adding the same endpoint to multiple NEGs causes undefined behavior.
The frontend configuration of a hybrid load balancer (like forwarding rules and URL maps) is standard. The special configuration applies only to the backend service. Envoy-based load balancers require an additional proxy-only subnet to host the Envoy proxies. Traffic flows from the load balancer's frontend, through the proxy layer (if present), to the backend service, which then directs it to the endpoints within the attached NEGs based on health status and balancing mode.
Choosing a NEG type involves a trade-off between management complexity and architectural flexibility. Managed NEGs with GKE Ingress automate setup but limit customization. Standalone or hybrid NEGs offer maximum flexibility for multi-environment architectures but require manual configuration of load balancer components, firewall rules, and health checks. The decision criteria include endpoint location, required load balancer type, and the need for mixed backends.
NON_GCP_PRIVATE_IP_PORT) allows Google Cloud load balancers to direct traffic to external endpoints in on-premises data centers or other clouds, reachable via Cloud VPN or Cloud Interconnect.Application Load Balancers use Open Request Cost Aggregation (ORCA) by reading backend load signals, such as CPU or memory utilization, reported inside HTTP response headers. The load balancer applies these signals to execute weighted policies, steering client requests away from overloaded instances and toward healthier backends.
A single backend service can only combine Google Cloud-based zonal Network Endpoint Groups (NEGs) and hybrid connectivity NEGs that point to external on-premises endpoints. No other mixture of backend types is permitted on a single backend service, and within Google Kubernetes Engine (GKE), this mixed configuration requires standalone NEGs.
GCE_VM_IP_PORT zonal NEGs route traffic directly to container Pod IP addresses and ports to provide container-native load balancing in Google Kubernetes Engine (GKE). In contrast, GCE_VM_IP zonal NEGs are used exclusively with internal and external passthrough Network Load Balancers to support features like GKE subsetting and traffic distribution across non-primary virtual machine network interfaces.
Last updated