Session Affinity in Cloud Load Balancing

Session affinity performance analysis evaluates what is gained and lost when a load balancer pins a client to the same backend. Affinity can lower latency for stateful work by reusing warm connections and local state, but it prevents even spreading of requests. The result is a tradeoff between faster per-client responses and less balanced use of resources across zones and regions.

1.1How Session Affinity Affects Global Distribution and Backend Use

Session affinity load imbalance happens when traffic flows from many clients to a fixed set of selected backends instead of to whichever backend has free capacity. When popular clients stick to the same managed instance group (MIG) or network endpoint group (NEG), those backends carry more requests while other backends sit underused. This creates uneven traffic patterns and hot instances that reach their target capacity sooner, which then forces new or returning sessions to break affinity and spill to other groups. Affinity improves continuity for an individual session but makes total resource use harder to predict at global scale.

Keeping a client on a nearby backend with existing state can reduce repeated setup work and round trips. Forcing a client to stay on a distant or busy backend can instead increase latency. Cloud Service Mesh (opens in a new tab) shows the same tension at larger scale: spreading traffic widely protects against local spikes but adds cross-zone and cross-region traffic, while keeping traffic local reduces network delay but creates uneven use.

1.2When Session Affinity Breaks Under Changing Capacity

Session affinity loss is the event where the load balancer can no longer send a client to its previously selected instance or endpoint. All session affinity options require that the selected backend stays configured and stays healthy, and affinity can break when the backend is removed, fails health checks, or becomes unreachable through the original path.

• The selected instance or endpoint must remain configured as a backend. Affinity breaks if you remove the selected instance from its instance group, remove the selected endpoint from its NEG, remove the instance group or NEG from the backend service, or if MIG autoscaling or autohealing removes the selected instance.
• The selected instance or endpoint must remain healthy. Affinity breaks when the selected instance or endpoint fails health checks and transitions from healthy to unhealthy or timeout.
• The routing path must remain stable for Global external Application Load Balancers and Classic Application Load Balancers. Affinity can break if a different first-layer Google Front End (GFE) serves later requests because the internet path from the client to Google changed.

Except for stateful cookie-based session affinity, affinity also depends on stable overall capacity. The instance group or NEG that holds the selected backend must not be full as defined by its target capacity, the total number of configured backends must remain constant, and the total number of healthy backends must remain constant. Adding or removing any instance, endpoint, or non-empty backend group changes the configured count, and any health transition between healthy and unhealthy changes the healthy count. Because fullness changes unpredictably in UTILIZATION balancing mode, use RATE or CONNECTION balancing mode to minimize breaks.

1.3How Spread and Locality Algorithms Trade Latency for Evenness

Cloud Service Mesh traffic distribution controls how client requests flow from each client to MIGs or NEGs in zones and regions. The spray to region algorithm spreads traffic from each client across all backend zones in a region, which reduces the risk of overloading MIGs or NEGs in a single zone during a rapid, localized spike. If zones A and B serve a region and zone B spikes, traffic is split between both zones instead of overwhelming zone B before the mesh can react. The cost is consistently higher cross-zone traffic even when local capacity remains, plus a larger affected area if two clients send to the same zones.

The spray to world algorithm spreads traffic from each client to all MIGs or NEGs globally across multiple regions. Cloud Service Mesh otherwise prefers the closest region to optimize latency, so spray to world is a larger spread radius for services that want uniform use across regions. All traffic flowing to all backends globally means a defective query can damage every deployment, and the extra cross-region traffic can increase request latency and create additional costs.

The waterfall by zone setting minimizes cross-zonal traffic by routing client traffic to the closest MIG or NEG in the zone up to its capacity, then to the next MIG or NEG in the same zone, and only then spilling to the next-closest zone. Traffic flows from the client to the nearest backend with available capacity, so overall latency may improve slightly because local backends are preferred. The tradeoff is uneven traffic across MIGs or NEGs within a region and higher sensitivity to single-zone spikes that are served entirely by one zone until the mesh reacts.

1.4How Preferred Backends and Capacity Draining Shape Resource Use

Preferred backends are a backend-service setting where Cloud Service Mesh sends client traffic to a designated group of backends first and uses them completely before routing to remaining backends. Traffic flows from clients to the preferred backends up to their configured capacity, and only excess traffic flows to non-preferred backends where the load balancing algorithm distributes it. One use is overflow to Google Cloud, where on-premises resources in a hybrid connectivity NEG are fully used before requests spill to autoscaled Google Cloud MIGs or NEGs, which minimizes cloud compute use while keeping failover capacity.

Automatic capacity draining is the protection that removes an unhealthy backend from global load balancing decisions so requests stop flowing to it. The service depends on health checks to detect failure, and traffic is then balanced among the remaining healthy backends to prevent overloading and optimize latency. This behavior is similar to setting capacity scaler to zero, and Cloud Service Mesh scales backend capacity to zero automatically when less than 25% of its individual instances or endpoints pass health checks.

Session affinity in Google Cloud load balancing is the practice of sending repeated requests from the same client to the same backend. The load balancer keeps a consistent client-to-backend mapping so state stored on one backend, such as a shopping cart or login session, remains available. This consistency improves performance for content delivery because the backend can reuse cached data and avoid rebuilding session state.

2.1Client IP Affinity

Client IP affinity is a session affinity type that maps a client to a backend based on the client's IP address. Traffic flows from the client to the load balancer, and the load balancer selects a backend using the source IP and sends later requests from that same IP to the same backend. Client IP affinity works without inspecting HTTP content, so it can support TCP and UDP flows where cookies or headers are not available. Choose Client IP affinity when clients have stable IP addresses and the application needs simple, network-level stickiness.

2.2Generated Cookie Affinity

Generated Cookie affinity is a session affinity type that maps a client to a backend using a cookie created by the load balancer. Traffic flows from the client to the Application Load Balancer (opens in a new tab), the load balancer generates a cookie on the first response, and the client sends that cookie back on later requests to return to the same backend. The cookie-to-backend mapping keeps HTTP(S) sessions stateful even when many clients share similar IP addresses. Choose Generated Cookie affinity for stateful web applications that run behind an Application Load Balancer and need reliable per-client stickiness.

2.3Header-Based Affinity

Header-based affinity is a session affinity type that maps a client to a backend using the value of a request header. Traffic flows from the client to the load balancer with a configured header, and the load balancer sends requests carrying the same header value to the same backend. The backend selection depends on the header value rather than the client IP or a load-balancer cookie, so different users or devices can be routed consistently. Choose header-based affinity when a stable header value identifies the session and the application needs stateful routing for HTTP(S) traffic.

2.4Affinity Across Application and Network Load Balancers

Affinity behavior depends on whether traffic uses an Application Load Balancer or a Network Load Balancer. Application Load Balancers include the global external Application Load Balancer, regional external Application Load Balancer, cross-region internal Application Load Balancer, and regional internal Application Load Balancer, and they process HTTP(S) requests where cookies and headers can be inspected. Network Load Balancers include the external passthrough Network Load Balancer, internal passthrough Network Load Balancer, classic proxy Network Load Balancer, and proxy variants for global external, regional external, regional internal, and cross-region internal traffic, and they handle lower-layer TCP, UDP, and proxy connections. The configuration requirement is to match the affinity type to the load balancer and the stateful requirement, so cookie and header affinity fit HTTP(S) workloads while Client IP affinity fits network-level workloads that cannot rely on HTTP content.

Design for resiliency and state persistence means treating session affinity as a performance optimization, not a guarantee of continuity. Cloud Load Balancing (opens in a new tab) can steer a client to the same backend, but that sticky routing can break during failures, scaling, or updates. A resilient design keeps backends replaceable and keeps session data in an external store so a different backend can continue the session.

3.1Keep Services Stateless When Possible

Stateless services handle each request independent of previous requests. A stateless service can grow, shrink, or restart without losing data needed for in-flight requests, because no required session data lives only in that instance's memory. Statelessness is especially important with an autoscaler, because the instances, nodes, or pods hosting the service can be created and destroyed unexpectedly.

It might not be possible for all services to be stateless. In that case, be explicit about which services require state and keep stateless and stateful services cleanly separated. That separation lets stateless services scale simply while stateful services use a more careful strategy with external persistence and affinity.

3.2Persist Session State Outside the Backend Instance

External persistence stores session data outside the selected backend instance so session continuity survives when affinity breaks. A cache increases data retrieval performance by reducing access to slower disk-based storage, and requests served from memory reduce latency and load on downstream databases. Cloud Spanner provides a strongly consistent, highly available, and globally distributed database with SQL support for session or application data that still needs relational features.

Choose storage by consistency, scalability, and availability needs, using different storage for different parts of the app when appropriate:

Web browsers retain the affinity cookie for the entire browsing session across multiple requests until the application is closed. Other clients treat a session as a single HTTP request and discard the cookie immediately after, so those clients cannot rely on cookie stickiness alone and depend even more on external state.

3.3Understand When Sticky Sessions Break During Failures and Scaling

Sticky session breaks happen when the selected backend can no longer serve the client. The selected backend instance or endpoint must remain configured as a backend and must remain healthy for affinity to hold.

• The selected backend must stay configured. Affinity can break when you remove the selected instance from its instance group, when managed instance group autoscaling or autohealing removes it, when you remove the selected endpoint from its network endpoint group (NEG), or when you remove its instance group or NEG from the backend service.
• The selected backend must stay healthy. Affinity can break when the selected instance or endpoint fails health checks and transitions from healthy to unhealthy or timeout.
• For Global external Application Load Balancers and Classic Application Load Balancers, the routing path must stay stable. Affinity can break if a different first-layer Google Front End (GFE) is used for later requests or connections after the internet routing path from the client to Google changes.

Except for stateful cookie-based session affinity, affinity has three additional stability requirements. First, the instance group or NEG containing the selected backend must not be full as defined by its target capacity, and for regional MIGs the zonal component must not be full. Because fullness can change unpredictably in UTILIZATION balancing mode, use RATE or CONNECTION balancing mode to minimize breaks. Second, the total number of configured backends must remain constant, and third, the total number of healthy backends must remain constant.

Any change in total configured or healthy backends can reshuffle sticky assignments. Adding includes adding instances to an instance group, autoscaling adding instances to a MIG, adding endpoints to a NEG, or adding a non-empty instance group or NEG to the backend service. Removing includes removing any instance or endpoint, autoscaling or autohealing removing any instance, or removing any non-empty instance group or NEG. Health changes also count, including any backend moving from unhealthy to healthy or from healthy to unhealthy or timeout.

3.4Protect the System From Cascading Failures

Cascading failure protection means one overloaded or failing service does not cause other services to fail. As service interdependencies grow, adopt patterns that reduce traffic to an overloaded service or handle errors gracefully, including the circuit breaker pattern, exponential backoffs, and graceful degradation. These patterns give an overloaded service time to recover or let the caller continue with reduced function instead of failing completely.

A service mesh links distributed services together and decouples business logic from networking. Traffic flows from one service through the mesh to the next service, and the mesh depends on health and routing signals to apply resiliency features such as request retries, failovers, and circuit breakers. Combined with stateless design and external state, the mesh helps sessions survive backend draining, updates, and autoscaling events.

Key Facts

• Session affinity improves per-client latency by reusing backend state but creates uneven traffic patterns and hot instances that use backend resources less evenly.
• Client IP affinity uses the source IP for TCP and UDP stickiness, Generated Cookie affinity uses a load-balancer cookie for HTTP(S) stickiness, and header-based affinity uses a request header value for HTTP(S) stickiness.
• Except for stateful cookie-based affinity, session affinity breaks when the selected backend is removed or unhealthy, when its group becomes full, or when the total number of configured or healthy backends changes.
• Cloud Service Mesh scales a backend's capacity to zero automatically when less than 25% of its instances or endpoints pass health checks.
• Resilient session design keeps services stateless when possible and stores required session state in an external store such as Cloud Spanner so a different backend can continue the session.