Kubernetes Health Checks for Application Availability

Optimizing probe configuration parameters for stability means setting Kubernetes timing and thresholds so real failures are found quickly without restarting or isolating healthy containers. In Google Kubernetes Engine (GKE), the readiness probe controls whether a Pod receives traffic, the liveness probe controls whether a container is restarted, and the startup probe blocks the other two until initialization finishes. The load balancer health check used by Ingress (opens in a new tab) runs outside the cluster, but it is derived from the readiness probe unless a BackendConfig custom resource supplies explicit settings.

1.1How Startup, Liveness, and Readiness Probes Divide Work

Probe roles in GKE divide availability work by the question each probe answers and the action Kubernetes takes on repeated failure. A startup probe asks whether the container has finished starting, a liveness probe asks whether a running container is stuck in an unrecoverable state such as a deadlock, and a readiness probe asks whether the instance should receive traffic right now. Data flows through a fixed gate: first the startup probe must succeed, then liveness and readiness checking begins.

Because passing startup is treated as ready to receive traffic, startup logic should also imply readiness. A new Pod should not receive requests before caches, connections, and frameworks inside the container are ready to serve them safely.

1.2Balancing Fast Detection Against Restart Churn

Balancing detection speed against churn means choosing timing values that react fast to deadlocks while tolerating slow startup and brief load spikes. The initialDelaySeconds setting delays the first probe until the application can realistically respond, the periodSeconds setting controls how often the kubelet re-checks the container, and the failureThreshold setting controls how many consecutive failures are required before Kubernetes restarts the container or removes the Pod from service. Short delays, frequent periods, and low thresholds detect failures faster, but they can turn a slow image pull, class load, cache warm-up, or traffic spike into repeated restarts and endpoint flapping.

Slow startup is a common cause of churn when liveness checking starts too early. Applications that load large caches or boot large frameworks can take minutes before an HTTP endpoint returns 200 OK. A startup probe protects this phase better than only lengthening the liveness delay, because liveness and readiness remain disabled until startup reports success. For readiness, the probe must report ready only after dependencies inside the container are ready, so scale-up and rolling updates do not send users to Pods that are still booting.

1.3Keeping Probe Endpoints Simple and Ports Consistent

Stable probe endpoint design keeps the check local, simple, and independent of other systems. A good default is a lightweight HTTP endpoint on the container that returns 200 OK when the container itself can serve, rather than a deep check of connection pools or downstream services. Probe logic must never call other services, because a slow or failed dependency would then drive restarts and traffic removal for a Pod whose own process is healthy.

Configuration consistency across Pod, Service, and Ingress also controls stability. The probe must target the correct path and port where the application actually listens, and the Pod containerPort must match the Service targetPort and the Ingress backend port or probes fail with errors such as connection refused. Pod probe fields cannot be edited in place, and changing a readiness probe for Pods already behind an Ingress does not automatically update the external load balancer health check. To apply a new probe definition, redeploy the Pods and the Ingress resource so GKE recreates the load balancer and its health checks with the new path, port, and protocol.

Diverse probe handlers are the different mechanisms the kubelet uses to check whether a container is healthy and ready to serve traffic. The kubelet sends a probe request from the node to the container, the container responds through its application protocol or process state, and the kubelet then marks the container as healthy or unhealthy based on that response. Choosing the handler that matches how the application actually communicates prevents false restarts and supports consistent observability (opens in a new tab) across web, network, microservice, and batch patterns.

2.1HTTP GET and TCP Socket Health Checks

HTTP GET probes check health by sending an HTTP GET request from the kubelet to a path and port inside the container. The request flows from the kubelet to the application web server, such as GET /healthz on port 8080, and success requires an HTTP status of 200 through 399. This handler verifies more than a listening port because the application must route the request, run health logic, and return a healthy code. Choose HTTP GET probes when the container serves HTTP or HTTPS traffic and can expose a dedicated health endpoint.

TCP socket probes check health by attempting to open a TCP connection from the kubelet to a container port. The connection attempt flows from the kubelet to the container network namespace, and success means the TCP handshake completes and the port accepts the connection. This handler sends no application data, so it cannot detect an application deadlock that still leaves the port open. Choose TCP socket probes when the workload uses non-HTTP protocols such as databases, message brokers, or raw TCP servers where an HTTP endpoint does not exist but a listening port proves the process is accepting work.

2.2gRPC and Exec Health Checks

gRPC probes check health by invoking the standard gRPC health-checking service from the kubelet to the container. The check flows from the kubelet as a gRPC client to the application gRPC server on its configured port, and success requires a SERVING status for the requested service. This handler understands gRPC semantics over Protocol Buffers and HTTP/2, so it tests the real request path for microservices. Choose gRPC probes when the architectural pattern is service-to-service gRPC, where an HTTP probe would miss the real path and a TCP probe would only confirm the port is open.

Exec command probes check health by running a command inside the container filesystem and namespaces. The execution flows from the kubelet to the container runtime, which starts the specified process such as cat /tmp/healthy or a custom script, and success requires an exit code of 0. This handler depends on tools present in the container image, so the probe fails if the binary or file is missing even when the application is healthy. Choose Exec probes when health cannot be observed over the network, such as batch workers, file-based initialization, or applications with custom internal state that only a local command can validate.

2.3Comparing Handlers and Their Success Thresholds

Probe handlers differ by what the kubelet sends, what counts as success, and which architecture they fit:

All handlers share common tuning that controls how quickly an endpoint changes state. CHECK_INTERVAL is the time in seconds from the start of one probe to the start of the next, with a default of 5. HEALTHY_THRESHOLD is the number of sequential passing probes required to consider the endpoint healthy, with a default of 2. UNHEALTHY_THRESHOLD is the number of sequential failing probes required to consider the endpoint unhealthy, with a default of 2. Together, the handler defines how health is tested and the thresholds define how many consistent results are needed before routing changes or a restart occurs.

Kubernetes health checking in Google Kubernetes Engine (GKE) uses three probes that divide container lifecycle work by the question asked. A startup probe asks whether the container has finished starting, a liveness probe asks whether the container is still healthy enough to keep running, and a readiness probe asks whether the container is ready to receive traffic. Together the probes provide self-healing through restarts and continuous availability through traffic control, with startup gating the other two during initialization.

3.1Startup, Liveness, and Readiness Roles

A startup probe determines whether the container application has started and is ready to accept traffic after long initialization. While the startup probe is running, the kubelet disables liveness and readiness checks so slow initialization does not trigger a premature restart or traffic failure. If the startup probe does not succeed within its allowed window, defined by failure threshold multiplied by probe period, the container is shut down and restarted instead of being left in an unknown starting state.

A liveness probe determines whether to restart a container that is already running. The kubelet sends the configured check, such as an HTTP GET to a health endpoint or a gRPC health request, and repeated failure beyond the failure threshold causes the container to be killed and restarted. Liveness probes are intended for unrecoverable failures where no other recovery is possible, such as a deadlock where the process is running but cannot make progress. A database replica stuck in a deadlock is a typical example, because restarting the replica is the only way to restore progress without affecting healthy replicas.

A readiness probe determines when a Pod should serve traffic through a Kubernetes Service. The readiness check begins only after the startup probe has succeeded, and the Pod is added to Service endpoints while the probe passes and removed from endpoints while the probe fails. A readiness failure does not terminate or restart the container, and traffic flows back to the Pod automatically when the probe starts passing again. This behavior supports temporary unavailability, such as a database replica that must pause request handling during a backup and then resume serving without a restart.

3.2Comparing Restart and Traffic Boundaries

The three probes differ by the lifecycle question they answer and by whether failure causes a restart or a traffic change:

The table shows a one-way dependency from startup to the other probes. Startup success unblocks traffic routing and steady-state monitoring, liveness drives the restart path, and readiness drives the traffic path without changing container state.

3.3Probe Order and GKE Ingress Health Checks

The three probes run in a strict order that protects starting containers and then separates restarts from traffic routing. First, the startup probe controls the starting phase and liveness and readiness checks remain disabled until the startup probe succeeds. Then, once startup succeeds, the container becomes ready and the Service begins sending traffic from the Service to the Pod endpoints. Finally, during steady-state operation, the liveness probe monitors container health for restarts while the readiness probe continuously adds or removes the Pod from endpoints as its ability to serve traffic changes.

A readiness probe in GKE also shapes traffic flow through external load balancing, not only inside the cluster. When a Service is exposed through a GKE Ingress (opens in a new tab), the Ingress controller creates a Google Cloud health check for each backend service that corresponds to a Kubernetes Service, and it derives path, port, and protocol from the serving Pods' readiness probe. This load balancer health check is different from the Kubernetes probe because the Kubernetes probe is executed by the kubelet inside the cluster while the load balancer check is executed from outside the cluster against the backend. A BackendConfig custom resource provides precise control over that external check and overrides the derived settings when tighter tuning is needed. If readiness settings change after Ingress creation, GKE does not automatically update the load balancer check, so Pods and the Ingress resource must be redeployed to recreate the load balancer and align the external check with the new readiness definition.

Key Facts

• A startup probe gates liveness and readiness checks until initialization succeeds; if startup never succeeds within failure threshold multiplied by probe period, the container is restarted.
• A liveness probe restarts a container after repeated failures to recover from unrecoverable states such as deadlock, while a readiness probe only removes a Pod from Service endpoints without restarting it.
• GKE Ingress derives the external load balancer health check path, port, and protocol from Pods' readiness probe unless a BackendConfig custom resource overrides it.
• HTTP GET probes succeed on HTTP status 200-399, TCP socket probes succeed when the connection is accepted, gRPC probes succeed on SERVING status, and Exec probes succeed on exit code 0.