Architecting a multi-NIC appliance deployment involves designing a hub-and-spoke network where a central routing VPC, containing a load-balanced group of network virtual appliances (NVAs), inspects and controls traffic flowing between multiple interconnected workload and services networks. The routing VPC acts as the hub, and traffic between spoke VPCs is forced through the NVAs for security inspection by using custom route advertisements and policy-based routing. Workload VPCs connect to this hub via VPC Network Peering, while a services-access VPC connects via HA VPN. This design ensures all traffic between different workload VPCs must flow through the central inspection point.
To establish connectivity, you create VPC Network Peering connections between the routing VPC and each workload VPC. On the routing VPC side, you enable Export custom routes, which advertises the paths to other networks. On each workload VPC side, you enable Import custom routes, which makes the NVA cluster in the routing VPC the next hop for inter-spoke traffic. For the services-access VPC, you establish an HA VPN connection to the routing VPC and use custom route advertisements on the Cloud Routers in both networks to announce specific subnet routes. This configuration allows transitive routing, enabling the services-access VPC to reach both external and workload networks through the routing VPC.
The network virtual appliances are deployed as a scalable, load-balanced group within the routing VPC. You create an instance template for the NVA image, which must have can IP forward enabled and requires at least two network interfaces: one attached to the routing VPC network and another for internet access. You then create a regional managed instance group (MIG) from this template. Finally, you set up an internal passthrough Network Load Balancer (ILB) with the instance group as its backend. This load balancer serves as the next-hop target for the custom routes, distributing traffic across the NVA instances.
To force traffic through the NVAs, you create routing controls in the spoke VPCs. In each workload VPC, you create a policy-based route with a destination range of 0.0.0.0/0 and a next-hop set to the internal load balancer's forwarding rule in the routing VPC. This overrides the default VPC routing, ensuring all outbound traffic from the workloads is directed to the NVA cluster for inspection before being routed to its final destination. The traffic flow is asymmetric; return traffic from the destination back to the source workload follows the same path through the routing VPC's NVAs, enforced by the imported custom routes.
High availability and elasticity for third-party network security appliances are established using internal passthrough Network Load Balancers paired with backend instance groups. Network appliances are deployed as backends within a managed instance group (MIG) or unmanaged instance group in a service producer VPC network. When traffic is selected for inspection via firewall policy rules in a consumer VPC network, it routes to an internal forwarding rule serving as the entry point to the load balancer, which distributes the flows across the appliance pool. To scale capacity and avoid traffic disruption, the load balancer continuously monitors backend instance health, dynamically routing traffic away from failed or degraded appliances.
High availability across multi-VPC interconnectivity architectures relies on coordinated route exchanges using Network Connectivity Center (NCC) and High Availability Virtual Private Network (HA VPN) gateways. Transit architectures terminate hybrid connections and consolidate routing across VPC spokes connected to an NCC hub, operating in either regional or global dynamic routing modes. When HA VPN gateways connect VPC networks directly, deploying both gateways in the same region provides a 99.99% availability service level agreement (SLA) using a dual-tunnel setup. Tunnel 0 connects interface 0 on the first HA VPN gateway to interface 0 on the peer HA VPN gateway, while Tunnel 1 connects interface 1 on the first HA VPN gateway to interface 1 on the peer. Deploying HA VPN gateways across different regions reduces the availability SLA to 99.9%. When scaling beyond a single gateway pair, an active-active routing configuration prevents the bandwidth degradation that occurs during failover in multi-gateway active-passive setups.
Operational monitoring for network virtual appliances and inter-VPC traffic integrates Google Cloud Observability (formerly Stackdriver), VPC Flow Logs, and Flow Analyzer. VPC Flow Logs captures 5-tuple network flow samplesâsource IP, destination IP, source port, destination port, and protocolâfrom VM network interfaces and routes them into Cloud Logging buckets. Flow Analyzer and Log Analytics query these log buckets using BigQuery engine capabilities, aggregating traffic by source and destination or client and server to track volume shifts, packet statistics, and anomalous flows. For Compute Engine instances hosting firewall appliances, the Google Cloud Ops Agent collects system metrics and detailed logs to monitor appliance resource consumption and health.
Tracking security rule hits, drops, and path reachability across multi-NIC inspection hops is managed via hierarchical firewall logging, security dashboards, and Connectivity Tests. Network Intelligence Center Connectivity Tests validates end-to-end trace paths between source and destination endpoints by analyzing configuration states and active data plane probing. The trace evaluates sequential checkpoints, verifying source IP spoof checks, hierarchical and VPC egress firewall rules, route table next-hop selection, and ingress firewall rules. When packets fail inspection or encounter mismatched routing configurations, the trace isolates the checkpoint failure type (such as blocked by a firewall rule or missing IP forwarding on intermediate multi-NIC appliances).
A multi-NIC virtual appliance serves as a bridge between VPC networks, with each network interface connected to a different VPC or subnet. The appliance must have IP forwarding enabled on its network interfaces to forward traffic between them, and instance-level firewall rules must be configured to allow the intended traffic flows. When traffic enters the appliance's untrusted interface, the NGFW inspects the packets and then forwards allowed traffic out the trusted interface to the destination VPC. Network tags and service accounts are used to identify which VM instances should have their traffic redirected to the inspection appliance through firewall policy rules. Each VPC network in this design must reside in the same project as the VM-based next-generation firewall (NGFW) because quotas are enforced at the project level.
Network Security Integration (NSI) provides an out-of-band method to integrate third-party inspection appliances with Google Cloud networks using a producer-consumer model. The service producer network contains scalable third-party appliances deployed as backends to an internal passthrough Network Load Balancer, and the consumer network uses firewall policies to select specific traffic and redirect it to an endpoint group. The endpoint group then sends the selected traffic to the producer, where the internal load balancer distributes it across the third-party appliances for inspection. This approach provides comprehensive visibility into workload traffic and supports advanced network security protections without requiring changes to routing policies.
When VPC networks with overlapping IP ranges need to communicate, Private NAT translates the source IP addresses to prevent conflicts. In a Network Connectivity Center topology, the Private NAT gateway is configured in one VPC to apply to all IP address ranges of a specific subnet, allowing VMs in overlapping VPCs to communicate by using the NAT IP ranges. The gateway reserves source ports for each VM in the networkâfor example, reserving 64 source ports per VMâand the NAT translation happens when packets traverse from the source VPC to the destination VPC through the hub.
Connectivity Tests validates that traffic can flow from a source VM to a destination VM by analyzing the network configuration path. The test performs a spoof check to verify the source VM can send packets with the specified source IP address, then evaluates egress firewall rules to confirm the source can send traffic, finds a matching route for the destination IP address according to routing order, and verifies ingress firewall rules allow the packet to arrive at the destination. If the destination IP address is a foreign IP address (not belonging to the destination VM), the destination VM must have IP forwarding enabled. The test result shows whether the packet could be delivered and includes details about which VPC firewall rules were evaluated in the path.
Flow Analyzer provides visibility into VPC traffic flows by analyzing VPC Flow Logs data stored in log buckets. It enables network traffic analysis with 5-tuple granularity (source IP, destination IP, source port, destination port, and protocol) without requiring complex SQL queries. Flow Analyzer organizes flow logs by source and destination or by client and server (where the initiator is determined by the smaller port number), and it supports time-range selection, basic filters, and SQL filters for detailed analysis. This tool helps verify that traffic is actually flowing through the inspection appliance as intended and identifies any unexpected traffic patterns.
Policy-based routing overrides the default VPC routing by specifying a destination range and next-hop target (the internal load balancer), while custom route advertisements export routes from the routing VPC to spoke VPCs so they know to send traffic to the NVA cluster. Policy-based routes are created in the spoke VPCs themselves, whereas custom routes are imported by the spokes from the routing VPC.
Private NAT translates source IP addresses to prevent conflicts when VPC networks have overlapping IP ranges. The gateway reserves source ports for each VM in the network (such as 64 ports per VM), and the NAT translation occurs when packets traverse from the source VPC to the destination VPC through the hub, allowing VMs in overlapping VPCs to communicate.
Professional Cloud Network Engineer
Prepare and test your skills
Prepare and test your skills
StudyCerts
StudyCerts