Container Service for Kubernetes:CPU QoS for containers

Limits

The following table describes the versions of the system components that are required to enable the CPU QoS feature for containers.

Billing rules

No fee is charged when you install and use the ack-koordinator component. However, fees may be charged in the following scenarios:

• ack-koordinator is an non-managed component that occupies worker node resources after it is installed. You can specify the amount of resources requested by each module when you install the component.

• By default, ack-koordinator exposes the monitoring metrics of features such as resource profiling and fine-grained scheduling as Prometheus metrics. If you enable Prometheus metrics for ack-koordinator and use Managed Service for Prometheus, these metrics are considered as custom metrics and fees are charged for these metrics. The fee depends on factors such as the size of your cluster and the number of applications. Before you enable Prometheus metrics, we recommend that you read the Billing topic of Managed Service for Prometheus to learn the free quota and billing rules of custom metrics. For more information about how to monitor and manage resource usage see Query the amount of observable data and bills.

Procedure

1. Create a file named configmap.yaml based on the following ConfigMap content:

   # Example of the ack-slo-config ConfigMap. 
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: ack-slo-config
     namespace: kube-system
   data:
     # Enable the CPU QoS feature. 
     resource-qos-config: |
       {
         "clusterStrategy": {
           "lsClass": {
             "cpuQOS": {
               "enable": true,
               "groupIdentity": 2
             }
           },
           "beClass": {
             "cpuQOS": {
               "enable": true,
               "groupIdentity": -1
             }
           }
         }
       }

   Specify lsClass and beClass to assign the LS and BE classes to different pods. cpuQOS includes the CPU QoS parameters. The following table describes the parameters.

   Parameter	Type	Valid value	Description
   enable	Boolean	true false	true: enables the CPU QoS feature for all containers in the cluster. false: disables the CPU QoS feature for all containers in the cluster.
   groupIdentity	Int	-1~2	Specify group identities for CPU scheduling. By default, the group identity of LS pods is 2 and the group identity of BE pods is -1. A value of 0 indicates that no group identity is assigned. A greater group identity value indicates a higher priority in CPU scheduling. For example, you can set cpu.bvt_warp_ns=2 for LS pods and set cpu.bvt_warp_ns=-1 for BE pods because the priority of LS pods is higher than that of BE pods. For more information, see Group identity feature.

   Note

   If koordinator.sh/qosClass is not specified for a pod, the ack-koordinator component configures the pod based on the original QoS class of the pod. The component uses the BE settings in the preceding ConfigMap if the original QoS class is BE. The component uses the LS settings in the preceding ConfigMap if the original QoS class is not BE.

2. Check whether the ack-slo-config ConfigMap exists in the kube-system namespace.

   • If the ack-slo-config ConfigMap exists, we recommend that you run the kubectl patch command to update the ConfigMap. This avoids changing other settings in the ConfigMap.

     kubectl patch cm -n kube-system ack-slo-config --patch "$(cat configmap.yaml)"

   • If the ack-slo-config ConfigMap does not exist, run the following command to create a ConfigMap named ack-slo-config:

     kubectl apply -f configmap.yaml

3. Create a file named ls-pod-demo.yaml and copy the following content to the file:

   apiVersion: v1
   kind: Pod
   metadata:
     name: ls-pod-demo
     labels:
       koordinator.sh/qosClass: 'LS' # Set the QoS class of the pod to LS. 
   spec:
     containers:
     - command:
       - httpd
       - -D
       - FOREGROUND
       image: registry.cn-zhangjiakou.aliyuncs.com/acs/apache-2-4-51-for-slo-test:v0.1
       imagePullPolicy: Always
       name: apache
       resources:
         limits:
           cpu: "4"
           memory: 10Gi
         requests:
           cpu: "4"
           memory: 10Gi
     restartPolicy: Never
     schedulerName: default-scheduler

4. Run the following command to deploy the ls-pod-demo pod in the cluster:

   kubectl apply -f ls-pod-demo.yaml

5. Run the following command to check whether the CPU group identity of the LS pod in the control group (cgroup) of the node takes effect:

   cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-pod1c20f2ad****.slice/cpu.bvt_warp_ns

   Expected output:

   # The group identity of the LS pod is 2 (high priority). 
   2

6. Create a file named be-pod-demo.yaml and copy the following content to the file:

   apiVersion: v1
   kind: Pod
   metadata:
     name: be-pod-demo
     labels:
       koordinator.sh/qosClass: 'BE' # Set the QoS class of the pod to BE. 
   spec:
     containers:
       - args:
           - '-c'
           - '1'
           - '--vm'
           - '1'
         command:
           - stress
         image: polinux/stress
         imagePullPolicy: Always
         name: stress
     restartPolicy: Always
     schedulerName: default-scheduler

7. Run the following command to deploy the be-pod-demo pod in the cluster:

   kubectl apply -f be-pod-demo.yaml

8. Run the following command to check whether the CPU group identity of the BE pod in the cgroup of the node takes effect:

   cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b6e96c8****.slice/cpu.bvt_warp_ns

   Expected output:

   # The group identity of the BE pod is -1 (low priority). 
   -1

   The output shows that the priority of the LS pod is high and the priority of the BE pod is low. CPU resources are preferably scheduled to the LS pod to ensure the service quality.

FAQ

Is the CPU QoS feature that is enabled based on the earlier version of the ack-slo-manager protocol supported after I upgrade from ack-slo-manager to ack-koordinator?

In an earlier version (≤ 0.8.0) of the ack-slo-manager protocol, the alibabacloud.com/qosClass pod annotation is used.

ack-koordinator is compatible with the earlier versions of the ack-slo-manager protocol. You can seamlessly upgrade from ack-slo-manager to ack-koordinator. ack-koordinator is compatible with the earlier protocol versions until July 30, 2023. We recommend that you upgrade the resource parameters in an earlier protocol version to the latest version.