Professional Cloud Network Engineer

Engineer Traffic Paths and Resolve Hybrid Routing Conflicts

Hybrid network routing controls how traffic flows between Google Cloud Virtual Private Cloud (VPC) networks and external networks, like on-premises data centers, over connections such as High Availability VPN (HA VPN) and Cloud Interconnect. The goal is to optimize path selection for performance and resilience while preventing issues like asymmetric routing or routing loops. This is achieved by configuring the Cloud Router to manage custom-advertised and custom-learned routes, using Border Gateway Protocol (BGP) attributes to influence routing decisions.

Active-Active and Active-Passive Routing Configurations

The choice between active-active and active-passive configurations determines how traffic is distributed across redundant tunnels. In an active-active setup, both VPN tunnels carry traffic simultaneously using equal-cost multipath (ECMP) routing, combining the bandwidth of both tunnels. The peer gateway and Cloud Router advertise routes with identical Multi-Exit Discriminator (MED) values, giving them equal priority. In an active-passive setup, only one tunnel carries traffic while the other acts as a standby, halving the effective throughput under normal conditions. The peer gateway advertises routes with different MED values, so the Cloud Router and peer gateway send all traffic through the tunnel with the highest priority (lowest MED).

Controlling Path Selection with MED and Route Priorities

The Multi-Exit Discriminator (MED) is a BGP attribute used to signal a preferred ingress path into a network. In hybrid routing, the on-premises router sets MED values on the routes it advertises to Google Cloud. The Cloud Router uses these MED values to assign priorities to the learned custom dynamic routes within the VPC, with lower MED values indicating higher preference. For egress traffic from Google Cloud, the route with the highest priority (lowest MED) is used. This mechanism directly controls path selection and must be set symmetrically on both sides to prevent asymmetric routing, where traffic takes one path out and a different path back, which can cause routing loops.

Failover Behavior and Route Withdrawal

Failover is triggered when a VPN tunnel becomes unhealthy or its BGP session goes down. The Cloud Router manages this by withdrawing the learned custom dynamic routes whose next-hop is the unavailable tunnel. This withdrawal can take 40–60 seconds, during which packet loss is expected. In an active-active configuration, traffic shifts to the remaining healthy tunnel using ECMP. In an active-passive configuration, when the highest-priority route is withdrawn, the Cloud Router automatically imports the next best route from the standby tunnel. A BGP session down event causes route removal even if the tunnel interface is still up, ensuring the routing table is updated.

Resolving Conflicts and Optimizing Topologies

To prevent asymmetric routing and loops, MED values and route priorities must be set symmetrically on both sides of the connection. For complex topologies with more than two HA VPN tunnels or multiple gateways, an active-passive configuration is not recommended. If configured across multiple gateways, the passive tunnels on a gateway will not be used for failover until all active tunnels on all gateways have failed, which can lead to unexpected bandwidth loss. Active-active designs or carefully planned failover domains are preferred.

Implement Custom Learned Routes for Hybrid Topologies

Custom learned routes are user-defined IP address prefixes configured directly on a Cloud Router BGP session. They dynamically inject external network destinations, like on-premises or multi-cloud CIDR ranges, into a VPC routing table without requiring the remote BGP peer to advertise them. This allows administrators to programmatically direct hybrid traffic without changing peer router configurations.

Custom Learned Routes Configuration and Connectivity

Custom learned routes are applied to specific BGP sessions on a Cloud Router across Cloud Interconnect VLAN attachments, Cloud VPN tunnels, or Network Connectivity Center Router appliance instances. Configuration can be done via the Google Cloud console, the gcloud CLI, or the Compute Engine API. In the CLI, prefixes are defined using commands like gcloud compute routers update-bgp-peer with flags such as --add-custom-learned-route-ranges. IPv4 addresses without a prefix length are interpreted as /32, and IPv6 as /128.

Route Evaluation, Priority Metrics, and Precedence

Cloud Router assigns a Multi-Exit Discriminator (MED) priority metric to custom learned routes, ranging from 0 to 65535, with lower values indicating higher preference. A default MED of 100 is applied if not specified. When multiple paths to the same destination exist, the VPC control plane selects the route with the lowest MED. Local VPC subnets always take precedence over dynamic routes; any custom learned route that is equal to or more specific than an existing VPC subnet range is ignored. Custom learned routes cannot be processed by BGP route policies.

System Quotas, Monitoring, and Operational Verification

Google Cloud enforces quotas to maintain routing performance. A single BGP session supports a maximum of 10 custom learned routes. Each Cloud Router can accept up to 5,000 dynamically advertised prefixes from a single BGP peer before resetting the session. Operational verification is done using CLI commands like gcloud compute routers describe to see configured prefixes and gcloud compute routers get-status to see the optimal dynamic routes currently active in the VPC data plane.

Configure and Manage Custom Advertised Routes

Custom advertisement mode in Cloud Router controls which IP prefixes are advertised to on-premises networks over BGP. Default mode only advertises local VPC subnets. Custom mode allows the advertisement of external IP ranges, arbitrary internal IPs, or IPs from other peered VPCs, enabling flexible hybrid routing topologies.

Advertisement Modes at Router and BGP Session Levels

Advertisement mode can be set at two levels: the entire Cloud Router or individual BGP sessions. The router-level setting applies to all its sessions unless overridden. A session-level setting of default mode means it inherits the router's configuration. A critical rule is that if any custom advertised route is specified on a BGP session, that session advertises only its custom prefixes and ignores all routes from the router-level configuration.

Effective Advertisement Behavior

The interaction between levels determines what is advertised. If both router and session use default mode, the session advertises local subnet ranges. If the router uses custom mode but the session uses default, the session inherits the router's custom prefixes. If the session uses custom mode—regardless of the router's mode—it advertises only its own custom prefixes. This granular control allows precise routing announcements per session.

Custom Advertisement Options

When configuring custom advertisement, you have two options. The first advertises only the custom IP ranges you specify, with a MED matching the BGP session's base priority. The second advertises both your custom ranges and the local VPC subnet ranges. For the second option, the MED for local subnets depends on the VPC's dynamic routing mode, while the MED for custom ranges matches the session's base priority.

Prefix Limits and IP Address Handling

Key limits include a maximum of 200 custom advertised routes per BGP session. If custom routes are identical across all sessions on a router, this limit applies to the total unique routes for the router. Cloud Router accepts a maximum of 5,000 prefixes from a single BGP peer; exceeding this resets the session. IPv4 addresses without a mask are interpreted as /32, and IPv6 as /128. IPv6 ranges are only advertised in BGP sessions where IPv6 is enabled.

Key Facts

• The Multi-Exit Discriminator (MED) BGP attribute controls path selection in hybrid networks, with lower values indicating a higher preference for a route.
• Custom learned routes are user-defined prefixes configured on a Cloud Router BGP session to inject external destinations into the VPC routing table without peer configuration changes.
• Custom advertisement mode in Cloud Router allows the advertisement of external IP ranges and IPs from other VPCs, not just local subnets.
• A BGP session configured with any custom advertised route will advertise only those custom prefixes and ignore all router-level advertised routes.
• Key quotas include a maximum of 10 custom learned routes per BGP session, 200 custom advertised routes per BGP session, and 5,000 dynamically learned prefixes per BGP peer.

Frequently Asked Questions

What is the difference between active-active and active-passive VPN configurations?

In an active-active configuration, both VPN tunnels carry traffic simultaneously using equal-cost multipath (ECMP) routing, combining their bandwidth. In an active-passive configuration, only one tunnel carries traffic at a time, acting as a standby, which halves the effective throughput but reserves full bandwidth for failover.

When should I use custom learned routes instead of relying on BGP advertisements from my on-premises router?

Use custom learned routes when you need to inject specific external destinations (like partner networks or appliances) into your VPC routing table without being able to change or control the BGP advertisements coming from the on-premises or peer router.

How does the Cloud Router decide between a custom learned route and a dynamically learned BGP route to the same destination?

The Cloud Router uses the Multi-Exit Discriminator (MED) priority metric. Between any dynamic routes (including custom learned ones) to the same destination, the route with the smallest MED value is selected. If no MED is specified for a custom learned route, it defaults to 100.

What happens if I configure a custom advertised route on a BGP session?

If you configure any custom advertised route on a BGP session, that session will advertise only those custom prefixes. It will not advertise the local VPC subnets or any custom prefixes defined at the router level, overriding the router's configuration for that specific session.