Professional Cloud Developer

Integration of Custom and External Metrics

Using Diverse Metrics for Scaling

The Horizontal Pod Autoscaler (HPA) in Google Kubernetes Engine (GKE) typically scales Pods based on basic resource use like CPU and memory. However, it can also scale using more specific signals that better reflect your application's workload. These are divided into two types: Custom Metrics, which come from inside your Kubernetes cluster (like requests per second), and External Metrics, which come from outside services (like the number of messages in a Google Cloud Pub/Sub queue). Using these allows the HPA to react to real performance issues, not just high CPU.

Connecting Metrics to the HPA

To use custom or external metrics, you need the GKE Custom Metrics Adapter. This adapter acts as a bridge, allowing the HPA to read metrics stored in Cloud Monitoring. It often works with the Managed Service for Prometheus, which collects detailed application data. This setup is crucial for tracking complex signals like request latency or queue depth, which standard monitoring might miss.

Configuring and Testing Advanced Scaling

You define which metrics to use in the HPA's configuration file (manifest). You can mix different metric types, like Pod-based and External metrics, in the spec.metrics field to create a strong scaling strategy. To test if it works, you can run a load test and use the command kubectl get hpa --watch to see the HPA change the replica count in real time. This confirms your application can handle traffic spikes correctly.

Resource-Based Scaling and Pod Specification Requirements

Enabling HPA with Resource Definitions

For the HPA to work at all, you must define resource requests and limits in your Pod's specification. These values tell Kubernetes how much CPU and memory each Pod is expected to use and the maximum it can use. Without these definitions, the HPA cannot calculate the average utilization percentage it needs to decide when to scale.

Setting Targets for Performance

You configure target utilization thresholds for metrics like CPU and memory. This tells the HPA, "scale up when average usage goes above this percentage." It's also a best practice to use scaling policies and stabilization windows (explained in the next section) to control how fast the HPA adds or removes Pods. This prevents the system from overreacting to small, temporary changes in traffic.

Monitoring and Optimizing the Setup

You should regularly test your autoscaling configuration. Performing a load test by sending a burst of traffic verifies that the HPA increases Pods when needed. GKE also offers features like utilization-based load balancing through the Gateway API. By setting a maximum utilization threshold in a GCPBackendPolicy, traffic can be rebalanced before Pods become overloaded, improving overall performance and efficiency.

Scaling Behaviors and Stabilization Policies

Controlling Scaling Speed

The HPA uses a continuous control loop to check metrics and decide when to scale. You can configure scaling behaviors to control the speed, or velocity, of these changes. You set limits on how many Pods can be added or removed at once, either as a fixed number or a percentage of the current count. This prevents the cluster from making drastic, unstable changes.

Preventing Thrashing with Stabilization

A key setting is the stabilization window, especially for scaling down. This is a waiting period the HPA observes after a spike in demand ends. It ensures that a drop in traffic is sustained before removing Pods. This prevents thrashing, where Pods are rapidly created and deleted due to minor, fleeting fluctuations, which wastes resources and can hurt performance.

Working with Multiple Metrics

The HPA can evaluate multiple metrics simultaneously, like CPU usage combined with a custom metric for request queue length. When it does this, it calculates a scaling recommendation from each metric and chooses the highest value. This ensures the application has enough Pods to handle the worst bottleneck. You must monitor the HPA (using kubectl get hpa) to ensure all metrics are reporting correctly; an unknown status indicates a problem with the metrics pipeline.