Use a Local Disk Through LocalVolume Provisioner in a Kubernetes Cluster

By Kan Junbao

Introduction

Alibaba Cloud provides local disk configurations for some Elastic Compute Service ECS instances. Local disks have the advantages of low latency, high random IOPS, high throughput, and high cost effectiveness, and has great advantages in some applications with high performance requirements.

In the Kubernetes system, local disks can be used through HostPath, LocalVolume, and other types of PV.

HostPath: The volume itself does not contain scheduling information. If you want to fix each pod on a node, you need to configure scheduling information, such as nodeSelector, for the pod;
LocalVolume: The volume itself contains scheduling information, and the pods using this volume will be fixed on a specific node, which can ensure data continuity.

This document describes how to use the Alibaba Cloud local disk using LocalVolume.

Use LocalVolume Provisioner to automatically create LocalVolume, and the PV comes with the information about the node to which it belongs.
Create a PVC to consume LocalVolume. Pods using this PVC will be scheduled to the node specified by the PV.

How Provisioner Works

1) Introduction to Local Disks:

To learn more about ECS local disks, refer to Local disks

Local disks cannot be mounted or unmounted;

Local disks and ECS must coexist. The number of local disks is determined when ECS is created;

The type of the machine determines the number of local disks, which can be obtained through API;

By default, local disks are mounted from /dev/vdb, which may be different from custom images.

2) Mounting Principle:

Provisioner mounts local disks to the target directory at startup:

The number of local disks in this machine can be queried through API;

Local disks are mounted from the initiating device (/dev/vdb, by default) to the target directory in turn (the initiating device can be configured);

The device detects the file system. If the device does not format the file system, a file system is created;

Provisioner polls and checks the target directory in real time, and creates a PV based on the target directory list:

When a PV is created, StorageClass can support the configuration of reclaimPolicy and volumeBindingMode;

After LocalVolume PV is deleted, it is automatically created again;

PV name: local-pv-{node name}-{device Name}. The PV name will be added to the label;

Deploy Alibaba LocalVolume Provisioner

1) Create ConfigMap:

The configuration parameters of Provisioner are placed in the ConfigMap, provisioner-config:

vendor: optional. If it is alibabacloud, the Alibaba Cloud local disk is enabled;

hostDir: the mount directory of the local disk on the host;

mountDir: the mount directory of the local disk on the container after it is mounted to the host;

volumeMode: the mount mode, supporting file systems and blocks;

fsType: the file system type, which defaults to ext4;

deviceStartWith: optional. It defaults to vdb, the starting name of the local disk;

mkFSOptions: optional. It is null by default, indicating that the parameter after "mkfs. *" is performed;

mountOptions: optional. It is null by default, indicating that the parameter of "mount -o" is performed;

apiVersion: v1
kind: ConfigMap
metadata:
  name: local-provisioner-config
  namespace: kube-system
data:
  storageClassMap: |
    local-volume:
       vendor: alibabacloud
       hostDir: /mnt/disks
       mountDir: /mnt/disks
       blockCleanerCommand:
         - "/scripts/shred.sh"
         - "2"
       volumeMode: Filesystem
       fsType: ext4
       deviceStartWith: vdb
       mkFSOptions: ""
       mountOptions: "nodelalloc"

2) Create StorageClass

LocalVolume StorageClass does not actually go to Provisioner PV, but is used to define configuration details, such as reclaimPolicy and volumeBindingMode.

reclaimPolicy: It supports Retain and Delete;

volumeBindingMode: It supports Immediate and WaitForFirstConsumer.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-volume
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

3) Create a Provisioner Service

For deployment through Daemonset, set hostPID and hostNetwork to true;

The privileged permission is added;

Deployment through AK or STS is supported;

The AK-free STS function is supported;

The node needs to configure the RAM role information, and the role has access to the cloud disk.

When using the STS function, you do not need to configure ACCESS_KEY_ID and ACCESS_KEY_SECRET parameters;

To configure STS, see Create custom authorization policies

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: local-volume-provisioner
  namespace: kube-system
  labels:
    app: local-volume-provisioner
spec:
  selector:
    matchLabels:
      app: local-volume-provisioner
  template:
    metadata:
      labels:
        app: local-volume-provisioner
    spec:
      hostPID: true
      hostNetwork: true
      serviceAccountName: admin
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/plugins/local-volume-provisioner:v1.12-7802d35-aliyun
          imagePullPolicy: "Always"
          name: provisioner
          securityContext:
            privileged: true
          env:
          - name: MY_NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: ACCESS_KEY_ID
            value: ""
          - name: ACCESS_KEY_SECRET
            value: ""
          volumeMounts:
            - mountPath: /etc/provisioner/config
              name: provisioner-config
              readOnly: true
            - mountPath:  /mnt/disks
              name: local
              mountPropagation: "HostToContainer"
            - mountPath: /etc/kubernetes
              name: etc
      volumes:
        - name: provisioner-config
          configMap:
            name: local-provisioner-config
        - name: local
          hostPath:
            path: /mnt/disks
        - name: etc
          hostPath:
            path: /etc/kubernetes

Consume LocalVolume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myclaim
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: local-volume
kind: Pod
apiVersion: v1
metadata:
  name: mypod
spec:
  containers:
    - name: myfrontend
      image: nginx
      volumeMounts:
      - mountPath: "/data"
        name: mypd
  volumes:
    - name: mypd
      persistentVolumeClaim:
        claimName: myclaim

Community

Use a Local Disk Through LocalVolume Provisioner in a Kubernetes Cluster

Introduction

How Provisioner Works

1) Introduction to Local Disks:

2) Mounting Principle:

Deploy Alibaba LocalVolume Provisioner

1) Create ConfigMap:

2) Create StorageClass

3) Create a Provisioner Service

Consume LocalVolume

Read previous post:

Read next post:

Alibaba Container Service

You may also like

Comments

Alibaba Container Service

Related Products

Container Service for Kubernetes

ACK One

Cloud-Native Applications Management Solution

Local Public Cloud Solution