Prepare and test your skills
Prepare and test your skills
Fleet registration and multi-cluster identity federation is the process of joining different Kubernetes clusters into one Google Cloud fleet with shared identity and management. Registration creates a fleet membership record for each cluster, while federation lets workloads exchange Kubernetes credentials for Google Cloud credentials. Together they allow centralized configuration, policy, and secure access across staging, production, and external environments.
GKE attached clusters is the feature that registers Kubernetes clusters created outside Google Cloud into the Google Kubernetes Engine (GKE) management environment. An attached cluster can be any conformant Kubernetes cluster, including Amazon EKS clusters and Azure AKS clusters. Registration creates an AttachedCluster resource in the GKE Multi-Cloud API as a regional resource in a Google Cloud project, and all cluster metadata is stored in that region to support data residency requirements. Once attached, the cluster is operated through the Google Cloud CLI or Google Cloud console with the same fleet tooling as native GKE clusters.
Attaching a cluster unlocks fleet-wide management tools:
Fleet membership is the centralized record that makes a registered cluster part of a fleet. Each cluster receives a unique membership name within its fleet, and operators use that membership name for fleet operations instead of only the local cluster name. Operators verify registration with gcloud container fleet memberships list or gcloud container fleet memberships describe MEMBERSHIP_NAME. The fleet host project acts as the trust and management boundary for operations across multiple Google Cloud projects. Configuration, policy, and service mesh controllers depend on a healthy membership and secure tunnel, so a cluster without healthy membership cannot receive fleet-wide configuration.
Connect Agent is the workload deployed inside each registered cluster that connects back to the Connect API service on Google Cloud. The agent establishes a secure tunnel from the cluster to Google Cloud, so management traffic flows from the Google Cloud control plane through the tunnel to the cluster Kubernetes API server without requiring a direct inbound network path. The Kubernetes API server authorizes requests from the agent through two role-based access control (RBAC) policies for the agent service account: an impersonation policy that lets the agent send requests on behalf of the service account, and a permissions policy that defines allowed operations on other Kubernetes resources. Connect Gateway then uses this authenticated path to provide access to cluster and workload management, including access through the Google Cloud console.
Workload Identity Federation for GKE lets GKE workloads exchange Kubernetes credentials for Google Cloud credentials for secure service authentication. Enabling federation depends on the Kubernetes Engine Service Agent, which manages compute resources such as instances, disks, and load balancers. If that agent lacks the Kubernetes Engine Service Agent role (roles/container.serviceAgent), enabling fails with an error similar to Error waiting for updating GKE cluster workload identity config: DeployPatch failed.
Two Google-managed service agents support identity and node operations:
| Service agent | Identity pattern | Purpose |
|---|---|---|
| Kubernetes Engine Service Agent | service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com | Manages GKE compute resources and enables Workload Identity Federation for GKE |
| Kubernetes Engine Default Node Service Agent | service-PROJECT_NUMBER@gcp-sa-gkenode.iam.gserviceaccount.com | Supports node system workloads such as logging and monitoring |
Both agents must hold their corresponding roles on the project: roles/container.serviceAgent and roles/container.defaultNodeServiceAgent. Operators resolve PROJECT_NUMBER with gcloud projects describe PROJECT_ID and verify grants with gcloud projects get-iam-policy PROJECT_ID. If a grant is missing, re-enabling the Kubernetes Engine API or adding the binding with gcloud projects add-iam-policy-binding restores it. In organizations that enforce the iam.automaticIamGrantsForDefaultServiceAccounts constraint, the default Compute Engine service account does not automatically receive expected permissions, so administrators must grant minimum least-privilege roles explicitly. That constraint is enforced by default for organizations created on or after May 3, 2024.
Exam tip: The container-engine-robot account needs roles/container.serviceAgent for Workload Identity Federation to succeed; the gcp-sa-gkenode account needs roles/container.defaultNodeServiceAgent for node system workloads.
Temporary support access (opens in a new tab) is a fleet-controlled way to grant Google Cloud Support read-only access to registered external clusters, including bare metal, VMware, GDC connected, GKE on AWS, GKE on Azure, and GKE attached clusters. Support access depends on a prior fleet membership and on the Connect Gateway API (connectgateway.googleapis.com) being enabled in the registration project. The operator must also hold gkehub.rbacrolebindings.create, which is included in the gkehub.editor and gkehub.admin roles.
Enabling support access follows an ordered workflow:
gcloud services enable connectgateway.googleapis.com --project=PROJECT_ID so fleet commands can reach the cluster.gcloud container fleet memberships support-access enable MEMBERSHIP_NAME --project=PROJECT_ID to create the dedicated support service account bindings.gcloud container fleet memberships support-access describe MEMBERSHIP_NAME --project=PROJECT_ID to confirm the access state.The dedicated support service account can then run read-only operations such as listing pods, checking container image pull success or failure, and inspecting node status, but it cannot change the cluster. Google removes the support permission after the support case closes, and operators can also revoke it manually when triage is complete.
Multi-cluster networking in Google Kubernetes Engine (GKE) combines east-west Pod-to-Pod communication across clusters with north-south external access through one global entry point. Multi-cluster Services (MCS) (opens in a new tab) provides cross-cluster service discovery and invocation inside a fleet, while Multi-cluster Ingress (MCI) (opens in a new tab) provides global external load balancing with failover across those clusters. Both depend on a GKE fleet as the shared membership and control boundary.
Multi-cluster Services (MCS) is the fleet feature that lets Pods discover and call Kubernetes Services running in other GKE clusters. When a Service is exported with MCS, that Service becomes available across all clusters in the fleet for highly available and geographically distributed applications. The initial export takes approximately five minutes to sync, while later endpoint updates sync immediately after the Service is established.
MCS automates three Google Cloud dependencies for discovery:
MCS use is included in the GKE cluster management fee with no extra MCS charge, and MCS does not incur Cloud Service Mesh endpoint charges even though the Traffic Director API must be enabled. Cloud DNS use incurs separate charges under Cloud DNS pricing.
Network and cluster requirements for MCS define where cross-cluster traffic can flow and which clusters can participate. Connectivity depends on clusters sharing the same Virtual Private Cloud (VPC) network or connected networks, and calls fail when clusters sit in separate, unconnected networks. While a fleet can span multiple projects and VPC networks, each exported multi-cluster service belongs to one project and one VPC network.
gke-mcs-importer Pod with elevated RBAC permission to manage endpoints, Services, and ServiceImports across the fleet, but the importer cannot view or modify Deployments.HttpLoadBalancing add-on enabled, and the add-on is enabled by default and should not be disabled.Enabling MCS in a fleet is the ordered process that registers clusters, creates identities, and opens permissions for endpoint propagation. The fleet host project holds the fleet, and enabling MCS there creates or ensures the service-PROJECT_NUMBER@gcp-sa-mcsd.iam.gserviceaccount.com service account exists. Required APIs include multiclusterservicediscovery.googleapis.com, gkehub.googleapis.com, cloudresourcemanager.googleapis.com, trafficdirector.googleapis.com, dns.googleapis.com, and connectgateway.googleapis.com.
gcloud container fleet multi-cluster-services enable in the fleet host project to activate service discovery for the fleet.roles/compute.networkViewer to the gke-mcs namespace importer identity so the gke-mcs-importer can read network and endpoint information from Cloud Service Mesh.gcloud container fleet multi-cluster-services describe and confirm the feature resourceState is ACTIVE and membership state reports firewall updates as OK.If clusters were registered with a Google Cloud service account instead of Workload Identity Federation for GKE, the node service account also needs roles/compute.networkViewer, and Compute Engine default service accounts need the compute.readonly and cloud-platform scopes. A newly created namespace can take up to five minutes before a previously exported Service is imported into it.
Exporting a Service with MCS is the act of creating a ServiceExport object that shares one Kubernetes Service with the fleet. The ServiceExport namespace must match the namespace of the Service being exported, and the ServiceExport name must match the Service name. MCS follows the Kubernetes multi-cluster services API, and MCS automatically creates a corresponding ServiceImport object in consuming clusters in the same namespace.
Consumption uses a fleet-wide DNS name of the form SERVICE_EXPORT_NAME.NAMESPACE.svc.clusterset.local, which resolves from any Pod in any fleet cluster. MCS only supports ClusterSetIP and headless Services, and only DNS A records are available. For ClusterSetIP Services, the name resolves to the ClusterSetIP recorded in the ServiceImport, while headless Services return endpoint addresses for direct Pod selection. Exporting the same Service name and namespace from multiple clusters groups them into one highly available multi-cluster endpoint with traffic distributed across clusters. Teams should avoid exporting Services in the default and kube-system namespaces where unintended name collisions are likely. When more than five Services share the same name and namespace, traffic distribution on imported Services may be limited to five exported Services.
Multi-cluster Ingress (MCI) is the centralized Ingress control plane that exposes applications in multiple clusters through one external load balancer virtual IP (VIP). MCI uses one config cluster as the central Kubernetes API server, and only one cluster can be the active config cluster at a time. Resources applied to other clusters are not processed by the Multi Cluster Ingress controller.
The deployment workflow moves traffic from the central definition to distributed Pod backends in order:
A MultiClusterService is a custom resource that selects Pods by labels and selects member clusters across the fleet, where member clusters are all clusters registered to the fleet. A MultiClusterService exists only in the config cluster and does not route traffic like a ClusterIP, LoadBalancer, or NodePort Service. Instead, the controller treats the MultiClusterService as one singular distributed resource that points to many cluster-local backends. For every MultiClusterService, the controller creates a derived headless Service and NEG in every selected target cluster, unless cluster selectors narrow the set. Only the selector and ports fields carry over to the derived Service, and the derived Service manages the NEG lifecycle for matching Pod endpoints. If no matching Pods exist in a target cluster, the derived Service and NEG remain empty, and users must not manage derived Services directly because the Ingress controller owns their lifecycle.
A MultiClusterIngress has the same schema and semantics as Kubernetes Ingress, except its backend.serviceName field references a MultiClusterService in the fleet API rather than a Service in one cluster. Traffic therefore flows from internet clients to the global load balancer VIP, then from the load balancer to NEG backends that point directly to Pod IP addresses across member clusters. Settings such as TLS termination are configured the same way as standard Ingress, and new load balancer deployment can take several minutes while updates to existing load balancers complete faster.
Exam tip: In MultiClusterIngress, backend.serviceName names a MultiClusterService, not a regular Kubernetes Service.
Comparing MCS and MCI shows how they divide multi-cluster traffic by direction and client location. MCS handles internal east-west calls between Pods, while MCI handles external north-south calls from clients outside the clusters. Both rely on fleet membership, but they create different network resources and failover behavior.
| Option | Primary client | Traffic path |
|---|---|---|
| Multi-cluster Services (MCS) | Pod in a fleet cluster calling name.namespace.svc.clusterset.local | Traffic flows from the calling Pod to the ClusterSetIP or Pod endpoints across clusters, with health tracked by the Traffic Director control plane |
| Multi-cluster Ingress (MCI) | External client calling the global load balancer VIP | Traffic flows from the client to the Compute Engine external load balancer, then to NEG backends that point to Pod IP addresses in member clusters |
Choose MCS when workloads inside the fleet need resilient service discovery without an external address. Choose MCI when external users need one stable VIP that survives cluster or regional failure because backends exist in multiple clusters.
Fleet-wide configuration management and GitOps-driven policy enforcement is the practice of operating many Google Kubernetes Engine (GKE) clusters as one fleet with consistent state and security. GKE attached clusters (opens in a new tab) lets teams register any conformant Kubernetes cluster, including Amazon EKS clusters and Azure AKS clusters, into the GKE management environment. Once attached, clusters are controlled through the Google Cloud CLI or Google Cloud console and gain access to fleet tools for configuration, policy, networking, and access.
Attaching a cluster gives centralized management across diverse staging and production environments:
Together, these tools let one fleet definition drive consistent state and security, while each member cluster can run in Google Cloud, on-premises, or in another cloud.
Config Sync is the fleet service that deploys cluster configurations across every member cluster to keep staging and production consistent. GitOps methodology with Config Sync means the desired cluster state is declared once for the fleet and then synchronized to each cluster, so drift in any environment is corrected toward the same declared state. The fleet configuration acts as the source and each attached or native GKE cluster acts as the destination for that synchronized state. Choose Config Sync when the problem is consistency at scale, where manual per-cluster changes would create differences between staging and production.
Policy Controller is the fleet service that automates policy and security across all clusters within the fleet. Policy Controller applies declarative security constraints and compliance rules consistently, so the same guardrails evaluate workloads in staging and in production. The policy definition lives at the fleet level and enforcement flows from the fleet control plane to each member cluster. Choose Policy Controller when the problem is preventing non-compliant configuration from running, rather than just distributing configuration files.
Exam tip: Config Sync deploys configurations across your fleet, while Policy Controller automates policy and security across all clusters within your fleet.
The AttachedCluster resource is the representation of an external Kubernetes cluster in the Google Cloud resource hierarchy within the GKE Multi-Cloud API. The AttachedCluster resource lets teams centrally operate configuration and control tools, including the Config Sync, Policy Controller, and Cloud Service Mesh products. When GKE attached clusters is installed, a new attached cluster resource is created within the Google project as a regional resource, and all metadata associated with the cluster is stored in the Google Cloud region associated with that resource. This regional storage permits compliance with data residency requirements because cluster metadata stays in the selected region.
The Connect Agent is the workload that GKE attached clusters deploys inside an external cluster to link it to Google Cloud. The Connect Agent connects back to the Connect API service on Google Cloud to create a secure link, and GKE attached clusters uses this secure tunnel to manage additional features for the cluster. The Connect Agent therefore sits inside the cluster network boundary while the Connect API service sits in Google Cloud, with management traffic flowing from Google Cloud through the tunnel to the cluster Kubernetes API server. Connect Gateway builds on this link to provide authentication and access to all clusters through a single interface, regardless of where the clusters run.
SERVICE_EXPORT_NAME.NAMESPACE.svc.clusterset.local, supporting only ClusterSetIP and headless Services with DNS A records.Config Sync deploys cluster configurations across every member cluster to keep staging and production consistent, while Policy Controller automates policy and security across all clusters within the fleet. Choose Config Sync when the problem is consistency at scale and Policy Controller when the problem is preventing non-compliant configuration from running.
Use Multi-cluster Services (MCS) when workloads inside the fleet need resilient service discovery without an external address, and use Multi-cluster Ingress (MCI) when external users need one stable virtual IP that survives cluster or regional failure. MCS handles internal east-west calls between Pods through fleet-wide DNS names, while MCI handles external north-south calls from clients outside the clusters through a Compute Engine external load balancer to Network Endpoint Groups that point to Pod IP addresses.
GKE attached clusters is the feature that registers conformant Kubernetes clusters created outside Google Cloud, including Amazon EKS clusters and Azure AKS clusters, as regional AttachedCluster resources in the GKE Multi-Cloud API with metadata stored in that region. Once attached, a cluster is operated through the Google Cloud CLI or Google Cloud console and gains access to Policy Controller, Config Sync, Cloud Service Mesh, and Connect Gateway.
Connect Agent is the workload deployed inside each registered cluster that connects back to the Connect API service on Google Cloud and establishes a secure tunnel from the cluster to Google Cloud. Management traffic then flows from the Google Cloud control plane through the tunnel to the cluster Kubernetes API server without requiring a direct inbound network path, and Connect Gateway uses this authenticated path to provide access through a single interface.
Your enterprise manages multiple Google Kubernetes Engine (GKE) clusters grouped into staging and production fleets. Your platform engineering team needs to implement a fleet-wide GitOps architecture using Config Sync and Policy Controller that satisfies the following requirements:
Which configuration architecture should you implement?
Last updated