Best practice for throttling block storage devices - Container Service for Kubernetes

Open source Kubernetes does not support throttling of block storage devices. Container Service for Kubernetes (ACK) allows you to throttle a block storage device that is mounted to a pod, such as a cloud disk or a local disk that is virtualized by using Logical Volume Manager (LVM). This helps you more efficiently manage and allocate storage resources, ensures system performance and stability, and reduces costs. This topic describes how to throttle a block storage device.

Limits

The OS used by the host must be Alibaba Cloud Linux 2 or later.
The Kubernetes version of your ACK cluster must be 1.20 or later.
The version of the Container Storage Interface (CSI) plug-in used by your cluster must be 1.22 or later. For more information about CSI versions, see csi-provisioner.

Parameters

readIOPS: specifies the maximum number of read operations per second that a pod can perform on a block storage device.
writeIOPS: specifies the maximum number of write operations per second that a pod can perform on a block storage device.
readBPS: specifies the maximum amount of data per second that a pod can read from a block storage device.
writeBPS: specifies the maximum amount of data per second that a pod can write to a block storage device.

How to throttle a block storage device

The following example shows how to throttle a cloud disk. The cloud disk can be mounted as a dynamically provisioned volume or a statically provisioned volume.

Note

Throttling settings cannot be changed after they are configured.

Mount the disk as a dynamically provisioned volume

Create a file named alicloud-disk-topology-essd.yaml and copy the following content to the file:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-disk-topology-essd
parameters:
  type: cloud_essd
  readIOPS: "100"
  writeIOPS: "10"
  readBPS: "100k"
  writeBPS: "100m"
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Run the following command to create a StorageClass:
```
kubectl apply -f alicloud-disk-topology-essd.yaml
```

Create a file named nginx.yaml and copy the following content to the file:

The application uses a persistent volume claim (PVC) that references the StorageClass you created to dynamically provision a cloud disk as a volume.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-csi-encrypt
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  podManagementPolicy: "Parallel"
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      hostNetwork: true
      containers:
      - name: nginx
        command:
        - sleep
        - "999999999"
        image: nginx
        volumeMounts:
        - name: disk-csi
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: disk-csi
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: alicloud-disk-topology-essd
      resources:
        requests:
          storage: 80Gi

Run the following command to create the application:
```
kubectl apply -f nginx.yaml
```

Run the following command to view the read limits and write limits of the block storage device used by the container:

Log on to the host. Specify the UID of the pod in the following code to verify that the limit is set to the value that you specified. 
readIOPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.read_iops_device
writeIOPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.write_iops_device
readBPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.read_bps_device
writeBPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.write_bps_device

Mount the disk as a statically provisioned volume

Use the following template to create a file named pv-static.yaml:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: d-****
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 80Gi
  csi:
    driver: diskplugin.csi.alibabacloud.com
    fsType: ext4
    volumeAttributes:
      app: nginx
      type: cloud_ssd
      readBPS: 100K
      readIOPS: "100"
    volumeHandle: d-****
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

Run the following command to create a PV:
```
kubectl apply -f pv-static.yaml
```

Use the following template to create a pvc-static.yaml file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: nginx
  name: disk-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 80Gi
  volumeMode: Filesystem
  volumeName: d-****

Run the following command to create a PVC:
```
kubectl apply -f pvc-static.yaml
```

Create a file named nginx.yaml and copy the following content to the file:

The application uses the PVC that you created. The PVC is used to statically provision a cloud disk as a volume.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-disk
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: disk-pvc2
          mountPath: /data
      volumes:
        - name: disk-pvc2
          persistentVolumeClaim:
            claimName: disk-pvc

Run the following command to create the application:
```
kubectl apply -f nginx.yaml
```

Run the following command to view the read limits and write limits of the block storage device used by the container:

Log on to the host. Specify the UID of the pod in the following code to verify that the limit is set to the value that you specified. 
readIOPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.read_iops_device
writeIOPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.write_iops_device
readBPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.read_bps_device
writeBPS: /sys/fs/cgroup/blkio/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod{pod_uid}.slice/blkio.throttle.write_bps_device