Managing Google Kubernetes Engine (GKE) clusters across an enterprise (e.g., fleets)

Fleet Registration and Multi-Cluster Identity Federation

Fleet Registration is the process of grouping multiple GKE clusters from different Google Cloud projects under a single management umbrella, called a fleet. This allows you to manage clusters in staging, production, and development from one central place, applying consistent security policies, logging, and identity rules across all of them.

Several types of service accounts are key to making fleets work securely. The Compute Engine default service account handles basic system tasks on cluster nodes. The Google APIs Service Agent and the Kubernetes Engine Service Agent have special permissions to manage Google Cloud resources on behalf of your clusters. You can check these permissions using the gcloud projects get-iam-policy command. Fleet Workload Identity builds on this by providing a unified way for workloads in any cluster to securely access Google Cloud services without managing separate passwords or keys.

Managing permissions correctly is critical, especially across multiple projects. In newer organizations, a policy might prevent the default service account from automatically getting the permissions GKE needs. In these cases, an administrator must manually grant the necessary roles. When setting up fleets, you should follow the principle of least privilege, giving each service account only the specific, predefined roles it needs rather than broad, powerful roles. This ensures secure and controlled operations across your entire fleet of clusters.

Multi-Cluster Networking, Service Mesh, and Global Traffic Management

To connect applications across different clusters in a fleet, you use Multi-Cluster Services (MCS). MCS extends a standard Kubernetes Service so it can be discovered and used by Pods in other clusters. For this to work, the clusters must be registered to the same fleet, use compatible VPC networking, and have the proper permissions. When you export a service using a ServiceExport resource, MCS automatically sets up the needed Cloud DNS zones and firewall rules. Pods in other clusters can then find that service using a special domain name ending in svc.clusterset.local.

For handling internet traffic, Multi-Cluster Ingress (MCI) provides global load balancing. MCI gives you a single anycast VIP address that can route user traffic to your application running in clusters across different regions. It uses a central config cluster to define the ingress rules. Behind the scenes, MCI creates Network Endpoint Groups (NEGs) that dynamically track the healthy Pods in each cluster. If Pods in one region or cluster fail, the global load balancer automatically reroutes traffic to the healthy Pods in another location, providing failover.

You can fine-tune this traffic management using custom resources. A BackendConfig lets you control settings like health checks and how connections are closed for backend services. A FrontendConfig manages settings for the load balancer itself, such as SSL policies. Using these tools together allows enterprise platform teams to ensure consistent, reliable, and secure traffic routing for applications deployed across a global fleet of GKE clusters.

Fleet-Wide Configuration Management and GitOps-Driven Policy Enforcement

Managing many clusters requires a way to keep their configurations identical and secure. Fleet-wide configuration management uses a central system to apply settings—like network policies or resource limits—to every cluster in the fleet. This prevents configuration drift, where clusters become different over time and cause operational problems. A popular method for this is GitOps, which treats a Git repository as the single source of truth for all cluster configurations.

Config Sync is the tool that puts GitOps into action for GKE fleets. It continuously watches your Git repository. When you commit a change, like updating a deployment file, Config Sync automatically pulls that change and applies it to all the designated clusters. This ensures consistency, provides a full audit trail of changes in Git, and allows for easy rollbacks by simply reverting a bad commit.

While Config Sync applies the desired state, Policy Controller acts as a security guard. It uses the Open Policy Agent (OPA) framework to enforce rules you define, such as “all containers must use images from our approved registry” or “Pods cannot run with root privileges.” Policy Controller constantly monitors the fleet and will block any resource that violates these policies, even if Config Sync tries to deploy it. This creates a secure, self-healing system where platform teams can define governance in code once and have it enforced automatically across all staging and production environments.