Professional Cloud Network Engineer
A VPC-native Google Kubernetes Engine (GKE) cluster assigns IP addresses directly to Pods and Services from a Virtual Private Cloud (VPC) network's subnet ranges. This design uses alias IP ranges, where each Compute Engine node hosts an alias IP block from which its local Pods receive individual addresses. Traffic flows directly between Pods across VPC Network Peering connections and through Cloud VPN or Cloud Interconnect without requiring custom static routes or network address translation. This direct routing removes extra latency associated with routing hops and eliminates consumption of VPC static route quotas that limit routes-based clusters.
A VPC-native cluster requires four separate IP address ranges across its lifecycle. The node subnet provides the primary IP range for worker nodes and internal load balancers, while secondary IP ranges on that same subnet provide dedicated address space for Pods and Services. The GKE control plane runs in a Google-managed tenant project and requires a dedicated /28 RFC 1918 address range that must not overlap with any subnet in the VPC network peering group. Administrators can expand a subnet's primary IP range and attach additional secondary Pod ranges to an existing cluster, but the secondary range assigned to Services cannot be modified or expanded after creation.
Organizations facing internal address exhaustion can deploy VPC-native clusters using non-RFC 1918 private ranges or privately used public IP (PUPI) ranges. Non-RFC 1918 allocations typically use RFC 6598 shared address space (100.64.0.0/10) or Class E address space (240.0.0.0/4) for secondary Pod allocations. When a cluster uses a PUPI range, workloads can no longer route traffic to external internet hosts that use those identical public IP addresses because the VPC network treats those addresses as local destinations. Additionally, an internal passthrough Network Load Balancer must draw its IP address from the subnet's primary range, meaning that the primary range itself must use the non-RFC 1918 block if the load balancer requires one.
Shared VPC architecture allows a central host project to own network subnets while individual service projects run GKE clusters attached to those shared resources. To deploy a VPC-native cluster in a service project, the host project administrator grants the Compute Network User role to the service project's GKE service accounts. The cluster in the service project then consumes pre-defined primary and secondary IP ranges managed within the host project's subnet. This separation allows central network teams to control IP allocation and firewall rules while application teams manage cluster workloads independently.
A Private GKE cluster isolates cluster nodes by assigning them internal-only IP addresses, blocking direct inbound and outbound internet exposure. Administrators can expose the control plane through a public endpoint, a private endpoint, or both. When using private endpoints exclusively, management tools such as Cloud Build private worker pools must be peered directly into the VPC network or routed through an internal network proxy. Outbound traffic from Pods is controlled by the IP masquerade agent (IMA), which decides whether a packet keeps its native Pod IP address or undergoes Source Network Address Translation (SNAT) to appear as the node's primary IP address.
The primary IPv4 range of a subnet provides IP addresses for Compute Engine worker nodes, internal load balancers, and control plane endpoints. Google Cloud reserves four addresses in every primary subnet range (the first two and last two addresses), which makes them unusable for nodes. When sizing a primary subnet for a target node count N, the required subnet mask size S is calculated with the formula S = 32 - ⌈log2(N + 4)⌉. For example, a cluster requiring 900 nodes needs at least a /22 primary subnet range, which yields 1,020 usable addresses.
A secondary IPv4 address range supplies the IP blocks assigned to each node for its running Pods. In GKE Standard clusters, each node receives a /24 CIDR block by default (256 addresses) to support up to 110 Pods per node, providing a 2-to-1 address buffer that handles address churn during Pod lifecycle updates. GKE Autopilot clusters fix this limit at 32 Pods per node, automatically provisioning a /26 CIDR block (64 addresses) per node. Network administrators can optimize IP consumption in GKE Standard by adjusting maximum Pod limits during node pool creation. The secondary IPv4 range for Services assigns internal, virtual cluster addresses (ClusterIP) to expose internal workloads. Modern GKE Standard and Autopilot clusters automatically default to a Google-managed Service range (34.118.224.0/20), which avoids consuming the organization's private VPC IP addresses.
Cross-project Identity and Access Management (IAM) configuration in GKE establishes the access permissions that service agents in a cluster project need to consume network and security resources in another project. A Shared VPC design separates the centralized network host project from individual service projects where clusters run. The Service Account Token Creator and Service Account User roles allow cluster management systems to authenticate node workloads using identity accounts from outside the cluster project. When a GKE node uses a custom service account located in a separate project, the Compute Engine service agent requires the Service Account Token Creator role on that custom service account to generate operational tokens, and the GKE service agent requires the Service Account User role on that same custom account to impersonate it for workload authentication.
Shared VPC networking permissions authorize service project accounts to provision compute instances that connect directly to host project subnets. Service project service accounts require the Kubernetes Engine Host Service Agent User (roles/container.hostServiceAgentUser) role at the host project level and the Compute Network User (roles/compute.networkUser) role at the host project or subnet level. The Host Service Agent User role gives the service project GKE service agent permission to coordinate networking with the host project GKE service agent. The Compute Network User role allows GKE nodes in the service project to bind to the shared subnet and consume its IP address space.
Workload Identity Federation for GKE links Kubernetes service accounts inside a cluster to IAM service accounts that can exist in external projects. This relationship lets containerized workloads authenticate to Google Cloud services without storing long-lived service account keys in the cluster. Administrators bind the Kubernetes service account to the IAM service account by granting it the Workload Identity User role on that IAM account. The IAM service account retains only the specific permissions needed for the target resources, maintaining strict security boundaries across projects.
Subnet topologies in GKE define how IPv4 address space is divided among cluster nodes, Pods, and Services inside a Virtual Private Cloud. VPC-native clusters use alias IP address ranges to allocate routable IP addresses directly from primary and secondary subnet CIDR blocks defined in the Shared VPC host project. Secondary IP ranges provide dedicated CIDR blocks within a subnet for Kubernetes Pods and ClusterIP Services. In a Shared VPC deployment, network administrators must create both secondary ranges in the host project before a cluster can be created in the service project. Subnets support up to 30 secondary IP ranges, and GKE defaults to allocating a /14 block for Pods and a /20 block for Services if custom ranges are omitted.
A private GKE cluster isolates compute workloads from the public internet by assigning private internal IP addresses to worker nodes and providing a private control plane endpoint. Deploying private clusters in a Shared VPC architecture centralizes network security policy in the host project while allowing workloads in service projects to communicate across the internal network. A Shared VPC architecture consists of a centralized host project that shares network infrastructure with one or more attached service projects. The host project owns the VPC network, subnets, secondary IP ranges, and firewall rules used by clusters in the service projects. Standard clusters using Shared VPC must be VPC-native, whereas Autopilot clusters are VPC-native by default. Shared VPC configuration must be established during cluster creation, as existing clusters cannot be converted to use Shared VPC.
Host project firewall rules define the security boundary for GKE traffic across nodes, Pods, and Services in all attached service projects. Central network administrators manage these rules to govern inbound and outbound traffic flows based on IP ranges, protocols, and ports. When Kubernetes creates external resources like load balancers, the host project must have matching firewall rules to permit traffic. If Kubernetes lacks host project permissions to create firewall rules automatically, it records events alerting administrators to create the rules manually.
Outbound internet traffic and access to Google APIs from private cluster nodes require specialized gateway configurations in the Shared VPC. Cloud NAT provides outbound internet connectivity for private nodes and Pods by translating internal IP addresses to external gateway addresses without exposing nodes to inbound connections. Private Google Access allows private nodes and Pods to reach Google APIs and services using internal Google-owned addresses instead of traversing the public internet. While Private Google Access is enabled automatically for standard private clusters, it must be enabled manually on the host subnet when using Shared VPC.
Designing and allocating IP subnets for a private GKE cluster involves planning distinct, non-overlapping IP address ranges for the cluster's worker nodes, Pods, Services, and the control plane. In a VPC-native cluster, these addresses come from the VPC network's subnets, ensuring all components can communicate directly without conflicting with other connected networks like on-premises systems. The primary IP address range of a chosen VPC subnet supplies addresses for the cluster's worker nodes and internal load balancers. Pods receive their IP addresses from a secondary subnet range configured as alias IPs on their node's network interface. Kubernetes Services like ClusterIPs get virtual IP addresses from a separate secondary subnet range.
A private GKE cluster's control plane runs in a Google-managed project and requires a dedicated /28 master IPv4 CIDR block. This small subnet provides 16 IP addresses reserved exclusively for the control plane's internal instances. You must select an unused private IP range (like RFC 1918 space) that does not overlap with any existing subnets in your VPC or any networks connected via peering or VPN. Although GKE checks for overlaps in a standalone VPC, in a Shared VPC you must manually coordinate with network administrators to ensure this /28 range is unique across all interconnected environments.
Pod density settings determine the size of the CIDR slice assigned to each individual node from the Pod secondary range. Standard clusters assign a /24 range (256 addresses) per node by default to accommodate up to 110 Pods. You can configure a larger /23 range per node for high-density workloads needing up to 256 Pods, or a smaller /26 range per node for Autopilot clusters or to conserve IP space. When sizing your subnets, you must account for future growth from the cluster autoscaler and node upgrades. If you expand a subnet after cluster creation, you must update the master authorized networks list to include the new range so new nodes can register with the control plane.
The IP masquerade agent on GKE nodes manages Source Network Address Translation (SNAT) for Pod traffic leaving the cluster. By default, it only masquerades a Pod's source IP for traffic destined to public internet addresses, changing it to the node's primary IP. Traffic to internal RFC 1918 destinations keeps the original Pod IP. If you use privately used public IP (PUPI) addresses for your Pod CIDR blocks to conserve private IP space, you must add these PUPI ranges to the agent's nonMasqueradeCIDRs configuration. This prevents SNAT on internal traffic, preserving end-to-end Pod IP visibility and preventing routing failures.
Resolving transitive and hybrid routing involves designing paths so clients in peered VPCs or on-premises networks can reach a private GKE cluster's control plane, despite the non-transitive nature of VPC Network Peering. VPC Network Peering is non-transitive, meaning traffic cannot flow through one peered connection to reach another. For example, if a GKE cluster's control plane is peered to a hub VPC, a spoke VPC also peered to that hub cannot automatically route traffic to the control plane. To enable access from a peered VPC or on-premises network, you can deploy a network proxy like a bastion host or an internal TCP/UDP load balancer within an authorized network subnet in the same VPC as the GKE cluster.
For hybrid access from on-premises networks connected via Cloud VPN or Cloud Interconnect, you can use Cloud Router custom route advertisements. You manually configure the Cloud Router to advertise the specific IP range of the GKE cluster's private control plane endpoint to your on-premises routers over BGP. This makes the control plane's IP address directly routable from on-premises, allowing traffic to flow over the hybrid connection without needing a proxy, assuming firewall rules permit it. Choose a proxy solution when the source network is another peered VPC where direct routing is impossible due to non-transitive peering. Choose custom route advertisements when the source is an on-premises network connected via VPN or Interconnect, as this provides a direct and simpler routing path.
Configuring master endpoint access modes determines how the Kubernetes API server is reachable and from which networks. You can choose to disable the public external endpoint entirely for maximum isolation, or enable it while restricting access to specific IP ranges via authorized networks. Private Service Connect (PSC) provides private connectivity to the GKE control plane without using the public internet. When enabled, Google Cloud assigns an internal IP address in your VPC that forwards requests to the cluster's management API. PSC supports scaling to many more clusters (up to 1000) compared to using VPC Network Peering (75 per zone/region). You have three primary configuration options for endpoint access: DNS-based only, IP-based only, or both combined.
GKE supports mixed-mode clusters where different node pools can have different accessibility settings. You can create private node pools (nodes with only internal IPs) or public node pools (nodes with external IPs). In Standard clusters, you configure this at the node pool level. In Autopilot clusters, you control it at the workload level by using a nodeSelector in your Pod specification (e.g., cloud.google.com/private-node=true to schedule Pods on private nodes). Node pool or workload-level network settings can override the cluster-level configuration.
Authorized networks are a GKE feature that controls which IP address ranges can access the cluster's control plane, which hosts the Kubernetes API server. When you enable a public endpoint, you can specify both public and private IP ranges. When you disable the public endpoint and use only a private endpoint, only private IP ranges can be used. Google manages firewall rules that enforce these restrictions, creating a network boundary that blocks traffic from untrusted IP addresses before it reaches the control plane. Preset addresses include Google-reserved IPs used for internal operations and GKE cluster IP ranges that enable communication between cluster components. Configurable access includes allowlisted external IP addresses, allowlisted internal IP addresses, and Google Cloud external IP addresses.
GKE offers a DNS-based endpoint that can be accessed from any network that can reach Google Cloud APIs. This endpoint relies on Identity and Access Management (IAM) and VPC Service Controls for security rather than source IP addresses. You control access by granting the gkehub.gateway.get IAM permission to users or service accounts. This creates an identity-aware policy where access depends on who the user is, not where they are connecting from. The DNS endpoint simplifies configuration and integrates well with broader Google Cloud security frameworks.
VPC Service Controls create a service perimeter around projects that host GKE clusters, governing data flows at the service level. This perimeter provides ingress and egress controls that restrict which identities and clients can communicate with the Kubernetes API server and other Google Cloud services from inside or outside the perimeter. When combined with authorized networks, VPC-SC adds a service-layer security boundary on top of the network-layer IP filtering, helping to prevent data exfiltration even if network access is obtained. The combined security approach uses authorized networks as the first line of defense by filtering traffic by IP address, IAM policies as the identity gatekeeper, and VPC Service Controls for the final service perimeter.
Authorized networks can be configured through the Google Cloud console or the gcloud CLI using specific flags. The --enable-master-authorized-networks flag enables the feature, while --master-authorized-networks specifies the comma-delimited list of CIDR values. The --enable-authorized-networks-on-private-endpoint flag restricts access to the internal endpoint to only authorized IP ranges, and --no-enable-google-access denies access from Google Cloud external IP addresses. The --enable-master-global-access flag allows access from IP addresses in other Google Cloud regions, which is important for multi-region deployments. You can specify up to 100 authorized IP address ranges combining both external and internal addresses.
Changes to certain authorized network settings may take several hours for GKE to propagate and enforce the firewall rule changes. If you expand a subnet used by a cluster with authorized networks configured, you must manually update the authorized network configuration to include the expanded IP address range, otherwise the new portion of the subnet will be blocked from accessing the control plane. For clients with dynamic IP addresses like home networks, frequent updates to the authorized network list are necessary. When using authorized networks with private clusters across different subnets, the Pod IP CIDR ranges from the calling cluster must be added to the authorized networks of remote clusters to allow Cloud Service Mesh control plane communication.
The DNS-based endpoint provides an alternative to IP-based control plane access by resolving the cluster endpoint using Cloud DNS infrastructure. This approach enables administrative access without requiring management of direct IP whitelists. Clients can connect from any network reachable by Google Cloud APIs, including on-premises environments and external clouds, without needing dedicated bastion hosts or proxies. Access through the DNS-based endpoint is secured using IAM policies and can be restricted at the perimeter using VPC Service Controls. This model simplifies multi-network administration by applying identity-aware security policies directly to control plane requests.
GKE Dataplane V2 is a networking dataplane implementation that provides enhanced network security and observability for GKE clusters. It automatically creates network policies and provides built-in network telemetry. When enabled, Dataplane V2 configures iptables rules on nodes to implement network policies and provides detailed flow logs for troubleshooting. This feature simplifies network policy implementation and provides deeper visibility into cluster network traffic patterns without requiring manual iptables configuration.
The IP masquerade agent on GKE nodes manages Source Network Address Translation (SNAT) for Pod traffic leaving the cluster. By default, it masquerades Pod source IPs for traffic destined to public internet addresses, changing them to the node's primary IP. Traffic to internal RFC 1918 destinations keeps the original Pod IP. Administrators can configure the nonMasqueradeCIDRs list to specify which destination ranges should not undergo SNAT, preserving the original source IP for internal routing. This configuration is critical when using non-RFC 1918 ranges like PUPI addresses to prevent routing failures.
GKE network policies control traffic flow between Pods at the network layer. They define which Pods can communicate with each other and with external resources. Network policies use pod selectors and namespace labels to identify traffic sources and destinations, and they support both ingress and egress rules. When network policies are applied, GKE configures iptables rules on nodes to enforce the defined restrictions. Network policies are essential for implementing zero-trust security models within the cluster.
Pod and Service CIDR ranges define the IP address space allocated to Kubernetes workloads. Pod ranges provide IP addresses to individual containers, while Service ranges provide virtual ClusterIP addresses for load balancing traffic across Pods. In VPC-native clusters, these ranges come from secondary IP ranges attached to the cluster's subnet. Administrators must carefully plan these ranges to avoid exhaustion and ensure they do not overlap with other network resources. The default Service range uses Google's managed 34.118.224.0/20 block to avoid consuming organizational IP addresses.
When a cluster exhausts its initial Pod address space, administrators can attach additional secondary ranges using discontiguous multi-Pod CIDR without destroying the cluster. New node pools can then be targeted to these new secondary ranges, allowing the cluster to scale beyond its original network footprint. This capability is essential for large-scale deployments that outgrow their initial IP allocation. Administrators must plan the timing and coordination of adding new ranges to ensure continuous cluster operations.
GKE clusters use a hierarchical DNS resolution system to resolve service names and external domains. Each node runs a local DNS cache (node-local DNS) to reduce latency and improve resilience. Kubernetes Services are registered with kube-dns, which resolves service names to ClusterIP addresses. For external DNS resolution, clusters can integrate with Cloud DNS for managed zone resolution. Administrators can configure custom DNS policies and stub domains to direct resolution to on-premises DNS servers or other custom resolvers when needed.
A VPC-native cluster uses alias IPs so that Pod addresses are native to the VPC network, allowing Pod traffic to route directly across VPC peering and hybrid interconnects without consuming VPC route quotas or introducing extra routing hops.
The service project's GKE service agent must be granted the Kubernetes Engine Host Service Agent User role (roles/container.hostServiceAgentUser) at the host project level. Additionally, the Compute Network User role (roles/compute.networkUser) must be granted to the service project GKE service agent and Google APIs service account on the specific shared subnet.
IP-based endpoints use authorized networks to filter traffic by source IP address, while DNS-based endpoints use IAM policies to control access based on user identity. DNS endpoints can be accessed from any network that reaches Google Cloud APIs without IP allowlisting.
If you assign a Privately Used Public IP (PUPI) range to your Pods, you must add that CIDR block to the nonMasqueradeCIDRs list in the IP masquerade agent's configuration. This tells the agent not to perform Source NAT on traffic destined for those IPs, preserving the original Pod source IP for correct routing within your internal network.
Prepare and test your skills
Prepare and test your skills
StudyCerts
StudyCerts