Optimizing backend traffic distribution, session affinity, and connection lifecycles ensures client requests are handled reliably across compute resources without dropping active sessions. Google Cloud uses Cloud Load Balancing, Cloud Service Mesh, and Google Kubernetes Engine (GKE) to spread load across backend types like Managed Instance Groups (MIGs) and Network Endpoint Groups (NEGs). Administrators configure policies for traffic splitting, locality-based balancing, capacity limits, and health checks to enable seamless failover and graceful termination of instances.
Locality load balancing policies determine how a load balancer distributes incoming client requests across available endpoints within a specific backend group or zone. In GKE, a resource called GCPBackendPolicy provides declarative control over load balancing modes and backend selection. Traffic can be routed using standard algorithms like round robin, least connections, random, or more advanced methods like locality and consistent hash.
For advanced workloads, Application Load Balancers can use real-time utilization metrics from backends to make smarter routing decisions. Backends send signals about their current loadāsuch as CPU or memory usageāinside HTTP response headers using the Open Request Cost Aggregation (ORCA) standard. The load balancer then uses these ORCA signals to execute weighted policies, ensuring traffic is steered away from overloaded instances and toward healthier ones.
Capacity management policies control how much traffic a backend service accepts before redirecting excess requests or triggering autoscaling. A backend service defines a capacity threshold, which is the maximum requests per second or resource utilization it can handle. When client demand exceeds this local capacity, the load balancer redirects the overflow traffic to the next closest healthy cluster or backend group in another zone.
Autoscaling systems integrate directly with these traffic metrics to dynamically add or remove compute resources. Workloads can be configured to autoscale based on HTTP requests per second or custom metrics, scaling up before performance degrades. Administrators can also configure explicit weight-based traffic splitting to divide traffic percentages across different backend versions or services.
Connection lifecycle management coordinates backend health checks, connection draining, and instance termination to prevent dropped connections during updates or failures. When a backend instance begins to fail its health checks or is being terminated, the load balancer enters a draining state for that endpoint. Draining allows in-flight requests to complete but stops sending new requests to that specific instance, immediately redirecting new traffic to other healthy backends.
Health checks continuously verify the operational state of backend instances using protocols like HTTP/1.1 and HTTP/2. Proper configuration requires aligning client protocols, proxy timeouts, and backend draining settings. This ensures zero downtime during instance autoscaling or unexpected faults. In multi-zonal deployments, precise firewall rules must allow health check probes from Google Cloud's IP ranges to reach all backend endpoints.
Multi-tier gateway architectures separate edge ingress management from internal routing policies by chaining managed load balancers with service mesh proxies. In a simple, single-level pattern, clients send requests directly to a Google Cloud-managed load balancer, which applies security policies and routes traffic to backend services.
In a more advanced two-level pattern, the managed load balancer forwards traffic to a secondary tier of Cloud Service Mesh edge proxies running on MIGs or GKE workloads. This two-level pattern provides operational separation: the first-level load balancer handles tasks like TLS termination and perimeter security, while the second-level edge proxy executes complex routing rules, such as matching HTTP headers. If incoming requests use protocols unsupported by the proxy layer, the traffic is dropped to protect the downstream services.
Managed Instance Groups (MIGs) serve as scalable backend populations for Google Cloud load balancers, distributing traffic across multiple, identical Virtual Machine (VM) instances that the platform automatically manages. A MIG is created from an instance template, and its controller maintains the desired number of instances, replacing any that become unhealthy. You can configure a MIG as either regional (spanning multiple zones for high availability) or zonal (confined to a single zone).
A backend service defines how a load balancer distributes traffic, and you attach MIGs or Network Endpoint Groups (NEGs) to it. The backend service configuration includes balancing modes that control distribution: rate mode distributes based on requests per second, utilization mode distributes based on backend capacity, and connection limits control the maximum concurrent connections per instance. When using NEGs, traffic can be sent directly to individual endpoints like container pods, which is useful for container-native load balancing in GKE environments.
Autoscaling policies determine when a MIG adds or removes instances based on workload demand. You can configure autoscaling using multiple signals monitored through Cloud Monitoring. CPU utilization-based autoscaling triggers scale-out when the average CPU usage across the group exceeds a set threshold. Load balancing serving capacity-based autoscaling uses the backend service's configured capacity limits to trigger scaling. For more precise control, custom metrics-based autoscaling allows scaling based on application-specific signals like request queue length.
Cool-down periods prevent rapid instance flapping by waiting a specified time after a scaling action before evaluating the need for another action, protecting against churn during brief traffic spikes. Scale-in controls determine how aggressively the MIG reduces capacity when traffic decreases. You also configure minimum and maximum instance limits to ensure sufficient capacity while controlling costs. The autoscaling algorithm uses a stabilization window to prevent rapid scale-down that could disrupt service, and these parameters can be adjusted based on your application's startup time and traffic patterns.
Network Endpoint Groups (NEGs) are logical groupings of endpointsādefined by an IP address and portāthat serve as backends for Google Cloud load balancers. Specialized NEG types allow you to integrate endpoints from diverse sources, including GKE containers, serverless platforms like Cloud Run, and external on-premises systems, into a unified load balancing setup.
A hybrid connectivity NEG is used to load balance traffic to endpoints outside Google Cloud, such as on-premises data centers or other public clouds, reachable via Cloud VPN or Cloud Interconnect. You create this NEG by specifying the endpoint type as NON_GCP_PRIVATE_IP_PORT and adding the external IP:Port combinations. You must choose a Google Cloud zone geographically close to your external environment to optimize latency. This NEG type can only be used with specific load balancers, and its balancing mode must be RATE for Application Load Balancers or CONNECTION for proxy Network Load Balancers.
You can configure a single backend service to reference a mixture of Google Cloud-based backends and external backends, but only in a specific combination. A backend service can include both zonal NEGs (for Google Cloud VMs or containers) and hybrid connectivity NEGs (for on-premises endpoints). No other mix of backend types is allowed. For GKE, this mixed-backend support is only available with standalone NEGs; Cloud Service Mesh does not support it. To configure this, you either create separate backend services for each type or use the single, mixed-backend service approach, ensuring the backend service protocol matches the load balancer type.
Zonal NEGs group endpoints within a single zone. The GCE_VM_IP_PORT type is primarily used for container-native load balancing in GKE, directing traffic directly to Pod IP addresses, which reduces latency and provides Pod-level visibility. GKE can manage these NEGs automatically, or you can create standalone NEGs for manual configuration.
The GCE_VM_IP zonal NEG type is used exclusively with internal and external passthrough Network Load Balancers. It enables flexible endpoint grouping, allowing a single VM to belong to multiple NEGs, and supports load balancing to non-primary network interfaces, which is useful for third-party appliances. In GKE, GCE_VM_IP NEGs enable subsetting for internal passthrough Network Load Balancers, improving scalability.
Health check configuration depends on the load balancer type. For NON_GCP_PRIVATE_IP_PORT endpoints used with global external Application Load Balancers, centralized health checks are required. This necessitates creating firewall rules on your external network to allow ingress traffic from Google's health check probe IP ranges (35.191.0.0/16 and 130.211.0.0/22).
For Envoy-based load balancers (like regional Application Load Balancers) using hybrid NEGs, distributed Envoy health checks are used. This requires a firewall rule to allow traffic from the region's proxy-only subnet to reach the external endpoints. With distributed health checks, every endpoint attached to a backend service must be unique; adding the same endpoint to multiple NEGs causes undefined behavior.
The frontend configuration of a hybrid load balancer (like forwarding rules and URL maps) is standard. The special configuration applies only to the backend service. Envoy-based load balancers require an additional proxy-only subnet to host the Envoy proxies. Traffic flows from the load balancer's frontend, through the proxy layer (if present), to the backend service, which then directs it to the endpoints within the attached NEGs based on health status and balancing mode.
Choosing a NEG type involves a trade-off between management complexity and architectural flexibility. Managed NEGs with GKE Ingress automate setup but limit customization. Standalone or hybrid NEGs offer maximum flexibility for multi-environment architectures but require manual configuration of load balancer components, firewall rules, and health checks. The decision criteria include endpoint location, required load balancer type, and the need for mixed backends.
StudyCerts
StudyCertsNON_GCP_PRIVATE_IP_PORT) allows Google Cloud load balancers to direct traffic to external endpoints in on-premises data centers or other clouds, reachable via Cloud VPN or Cloud Interconnect.A Managed Instance Group (MIG) is a group of identical Virtual Machine instances managed by Google Cloud, used as a backend for distributing traffic. A Network Endpoint Group (NEG) is a more flexible logical grouping of IP:port endpoints, which can represent individual containers in GKE, serverless functions, or even external on-premises systems, providing more granular load balancing.
Use a hybrid connectivity NEG when you need a Google Cloud load balancer to distribute traffic to backend endpoints located outside of Google Cloud, such as in an on-premises data center or another public cloud. This requires connectivity via Cloud VPN or Cloud Interconnect and specific configuration of firewall rules for health checks.
When an autoscaling event requires a backend instance to be terminated, the load balancer places that instance into a draining state. In this state, the instance stops receiving new connection requests but is allowed to finish processing its existing, in-flight requests for a configured timeout period. This prevents active client sessions from being dropped abruptly.
Container-native load balancing in GKE uses zonal NEGs of type GCE_VM_IP_PORT to allow Google Cloud load balancers to send traffic directly to the IP addresses of individual Pods, rather than routing through node-level proxies. This reduces latency, provides Pod-level health checking and metrics, and is more efficient for containerized workloads.
Professional Cloud Network Engineer
Prepare and test your skills
Prepare and test your skills