Container Service for Kubernetes:Implement horizontal pod autoscaling

Before you begin

To help you better use the HPA feature, we recommend that you read the Kubernetes official documentation Horizontal Pod Autoscaling to understand the basic principles, algorithm details, and scaling configurations of HPA before reading this topic.

In addition, Container Service for Kubernetes (ACK) clusters provide various workload scaling solutions for scheduling layer elasticity and node scaling solutions for resource layer elasticity. We recommend that you read the Auto scaling overview to understand the applicable scenarios and usage limits of different solutions before using the HPA feature.

Prerequisites

• An ACK managed cluster or ACK dedicated cluster is created. For more information, see Create a cluster.

• If you use the kubectl command to implement HPA, you must make sure that a kubectl client is connected to the Kubernetes cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Create an application that has HPA enabled in the ACK console

ACK is integrated with HPA. You can create an application that has HPA enabled in the ACK console. You can enable HPA when you create an application or for an existing application. We recommend that you create only one application that has HPA enabled for a workload.

Create an application that has HPA enabled by using kubectl

You can also create an HPA by using an orchestration template and associate the HPA with the Deployment for which you want to enable HPA. Then, you can run kubectl commands to enable HPA. We recommend that you create only one application that has HPA enabled for a workload. In the following example, HPA is enabled for an NGINX application.

1. Create a file named nginx.yml and copy the following content to the file.

   Important

   You must configure the request resources required by the application. Otherwise, you cannot enable HPA. You can use the resource profiling feature to analyze historical data of resource usage and get recommendations for configuring container requests and limits. For more information, see Resource profiling.

   Expand to view YAML template

   apiVersion: apps/v1 
   kind: Deployment
   metadata:
     name: nginx
     labels:
       app: nginx
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: nginx  
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx:1.7.9 # Replace it with your actual <image_name:tags>.
           ports:
           - containerPort: 80
           resources:
             requests:                         # This parameter is required for running the HPA.
               cpu: 500m

2. Run the following command to create an NGINX application:

   kubectl apply -f nginx.yml

3. Create a file named hpa.yml and copy the following content to the file to create an HPA.

   Use the scaleTargetRef parameter to associate the HPA with the nginx Deployment and trigger scaling operations when the average CPU utilization of all containers in the pod reaches 50%.

   YAML template for clusters whose Kubernetes versions are 1.24 and later

   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: nginx-hpa
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: nginx
     minReplicas: 1  # The minimum number of containers that must run for the Deployment. The value of this parameter must be an integer greater than or equal to 1.
     maxReplicas: 10  # The maximum number of containers to which the Deployment can be scaled. The value of this parameter must be greater than minReplicas.
     metrics:
     - type: Resource
       resource:
         name: cpu
         target:
           type: Utilization
           averageUtilization: 50 # The average utilization of the target resource, which is the ratio of the average value of resource usage to its request amount.
                  

   YAML template for clusters whose Kubernetes versions are earlier than 1.24

   apiVersion: autoscaling/v2beta2
   kind: HorizontalPodAutoscaler
   metadata:
     name: nginx-hpa
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: nginx
     minReplicas: 1  # Must be an integer greater than or equal to 1.
     maxReplicas: 10  # Must be greater than minReplicas.
     metrics:
     - type: Resource
       resource:
         name: cpu
         target:
           type: Utilization
           averageUtilization: 50
                  

   If you need to specify both CPU and memory metrics, you can specify both cpu and memory types of resources under the metrics field instead of creating two HPAs. If HPA detects that any one of the metrics reaches the scaling threshold, it will perform scaling operations.

   metrics:
     - type: Resource
       resource:
         name: cpu
         target:
           type: Utilization
           averageUtilization: 50
     - type: Resource
       resource:
         name: memory
         target:
           type: Utilization
           averageUtilization: 50

4. Run the following command to create an HPA:

   kubectl apply -f hpa.yml

   At this point, run the kubectl describe hpa <HPA name> command, a warning similar to the following output is returned, indicating that the HPA is still being deployed. The HPA name in this example is nginx-hpa.yml. You can run the kubectl get hpa command to check the status of the HPA.

   Warning  FailedGetResourceMetric       2m (x6 over 4m)  horizontal-pod-autoscaler  missing request for cpu on container nginx in pod default/nginx-deployment-basic-75675f5897-mqzs7
   
   Warning  FailedComputeMetricsReplicas  2m (x6 over 4m)  horizontal-pod-autoscaler  failed to get cpu utilization: missing request for cpu on container nginx in pod default/nginx-deployment-basic-75675f5

5. Wait for the HPA to be created and the pod to reach the scaling condition, which is when the pod CPU utilization of NGINX exceeds 50% in this example. Then, run the kubectl describe hpa <HPA name> command again to check the horizontal scaling status.

   If the following output is returned, the HPA is running as expected:

   Type    Reason             Age   From                       Message
     ----    ------             ----  ----                       -------
     Normal  SuccessfulRescale  5m6s  horizontal-pod-autoscaler  New size: 1; reason: All metrics below target

Related operations

The autoscaling/v2 API allows you to use the behavior parameter of HPA to configure scaling settings. You can specify the scaleUp and scaleDown fields in the behavior parameter to specify scale-out and scale-in settings.

Configure pods to only perform scaling operations

You can configure HPA to perform only scale-out operations or only scale-in operations by using the selectPolicy field. For more information, see Configurable scaling behavior.

• By default, scale-out and scale-in are not disabled.

• Disable scale-out: Set selectPolicy to Disabled in the scaleUp field. Example:

  behavior:
    scaleUp:
      selectPolicy: Disabled

• Disable scaling-in: Set selectPolicy to Disabled in the scaleDown field. Example:

  behavior:
    scaleDown:
      selectPolicy: Disabled

Configure the stable window

If the scale-in and scale-out metrics fluctuate frequently, you can add the stabilizationWindowSeconds field under the behavior parameter to set a stable window for a period of time, limiting the fluctuation of the number of replicas. This window can specify how long the HPA controller needs to observe the metric data before triggering scale-in or scale-out to determine the expected status of the system, preventing excessively frequent or unnecessary scaling due to short-term metric fluctuations.

In the example below, if the metric indicates that the target should scale down, the auto-scaling algorithm will consider all desired states in the past 5 minutes and use the maximum value within the window period.

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300

FAQs

• What do I do if unknown is displayed in the current field in the HPA metrics?

• What do I do if HPA cannot collect metrics and fails to perform scaling?

• What do I do if excess pods are added by HPA during a rolling update?

• What do I do if HPA does not scale pods when the scaling threshold is reached?

• How do I configure the metric collection interval of HPA?

• Can CronHPA and HPA interact without conflicts? How do I enable CronHPA to interact with HPA?

• How do I fix the issue that excess pods are added by HPA when CPU or memory usage rapidly increases?

• What do I do if HPA scales out an application while the metric value in the audit log is lower than the threshold?

• Can HPA determine the order in which pods are scaled in?

• What does the unit of the utilization metric collected by HPA mean?

• What do I do if unknown is displayed in the TARGETS column after I run the kubectl get hpa command?

• How do I find the metrics that are supported by HPA?

• How do I configure horizontal autoscaling after I customize the format of NGINX Ingress logs?

References

Other related documentation

• For more information about how to configure external metrics supported by Kubernetes to implement HPA based on Alibaba Cloud metrics, see Implement horizontal auto scaling based on Alibaba Cloud metrics.

• For more information about how to convert Managed Service for Prometheus metrics to metrics that are supported by the HPA to implement HPA, see Implement horizontal auto scaling based on Prometheus metrics.

• For frequently asked questions that you may encounter when you use HPA, see FAQs about node auto scaling for troubleshooting.

• For more information about how to enable CronHPA and HPA to interact without conflicts, see Make CronHPA compatible with HPA.

Other workload scaling solutions

• If your application resource usage has periodic changes and you need to scale pods regularly according to a Crontab-like strategy, see Use CronHPA for scheduled horizontal scaling.

• If your application resource usage has periodic changes that are difficult to define by rules, you can use Advanced Horizontal Pod Autoscaling (AHPA) to automatically identify resource usage cycles and scale pods based on historical metrics. For more information, see AHPA overview.

• If you need to automatically set resource limits for pods based on pod resource usage to ensure that pods get enough computing resources, see Use Vertical Pod Autoscaler.

• If you need to flexibly customize scaling policies for pods based on message queues, timing strategies, custom metrics, and other Kubernetes events, see ACK KEDA.

Combination solutions

You can use HPA with the node auto scaling feature to automatically scale nodes when cluster node resources are insufficient. For more information, see Enable node auto scaling.