Use CNFS to manage shared NAS volumes to address the issue of sharing data among pods - Container Service for Kubernetes

Container Service for Kubernetes (ACK) allows you to mount and use shared File Storage NAS (NAS) volumes that are managed by Container Network File System (CNFS). If you want to mount a directory in a NAS file system to multiple Kubernetes applications or pods, you can use CNFS to create a shared NAS volume. This solves the problem of sharing data among pods. This topic uses a StatefulSet as an example to describe how to use CNFS to manage shared NAS volumes.

Prerequisites

NAS is activated.
If this is the first time you visit the product page of NAS, follow the on-screen instructions to activate the NAS service.
A Container Service for Kubernetes (ACK) cluster that runs Kubernetes 1.20 or later is created. The Container Storage Interface (CSI) plug-in is used as the volume plug-in. For more information, see Create an ACK managed cluster.
The versions of csi-plugin and csi-provisioner are v1.24.11-5221f79-aliyun or later. For more information about how to update csi-plugin and csi-provisioner, see Install and update the CSI components.
The version of storage-operator is v1.24.105-825188d-aliyun or later. For more information about how to update storage-operator, see Manage components.
A kubectl client is connected to your cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Note
For more information about CNFS, see CNFS overview.
For more information about how to use CNFS to manage NAS file systems, see Use CNFS to manage NAS file systems (recommended).

Step 1: Create a workload for a shared volume

Create a persistent volume (PV) named cnfs-nas-static-pv and reference a CNFS named cnfs-nas-filesystem in the PV. The name of the CNFS is the same as the NAS file system that you created.
Create a persistent volume claim (PVC) named cnfs-nas-static-pvc and bind the PV to the PVC.
Create a StatefulSet named cnfs-nas-static-sts, use a BusyBox image to mount the PVC, and write a temporary file named 1G.tmpfile (1 GB in size) to the mount target.

Show YAML content:

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  name: cnfs-nas-static-pv
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
  csi:
    driver: nasplugin.csi.alibabacloud.com
    fsType: nfs
    volumeAttributes:
      containerNetworkFileSystem: cnfs-nas-filesystem # Reference the CNFS named cnfs-nas-filesystem. 
      mountProtocol: nfs
      path: /
      volumeAs: subpath
      volumeCapacity: "true"
    volumeHandle: cnfs-nas-static-pv
  mountOptions:
  - nolock,tcp,noresvport
  - vers=3
  persistentVolumeReclaimPolicy: Delete
  storageClassName: cnfs-nas-sc
  volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cnfs-nas-static-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: cnfs-nas-sc
  volumeMode: Filesystem
  volumeName: cnfs-nas-static-pv # Reference the PV named cnfs-nas-static-pv. 
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cnfs-nas-static-sts
  labels:
    app: busybox
spec:
  serviceName: "busybox"
  replicas: 2
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["/bin/sh"]
        args: ["-c", "sleep 3600;"]
        volumeMounts:
        - mountPath: "/data"
          name: cnfs-nas-static-pvc
      volumes:
      - name: cnfs-nas-static-pvc
        persistentVolumeClaim:
          claimName: cnfs-nas-static-pvc # Reference the PVC named cnfs-nas-static-pvc. 
EOF

Step 2: View the mount result

Run the following command to view the mount result:

kubectl exec cnfs-nas-static-sts-0 -- mount |grep nfs

Expected output:

971134b0e8-****.cn-zhangjiakou.nas.aliyuncs.com:/ on /data type nfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

The output indicates that the volume is mounted.

Step 3: Check whether data is persisted to the volume

Run the following command to write a temporary file that is 1 GB in size to the directory:

kubectl exec cnfs-nas-static-sts-0 -ti -- sh -c 'dd if=/dev/zero of=/data/1G.tmpfile bs=1G count=1;'

Run the following command to check whether the temporary file is written to the directory:

kubectl exec cnfs-nas-static-sts-0 -- ls -arlth /data

Expected output:

total 1G
-rw-r--r--    1 root     root        1.0G Dec 15 12:11 1G.tmpfile

The output indicates that the 1G.tmpfile file is written to the /data directory.

Step 4: Check whether the data is shared with other pods

Run the following command to check whether the temporary file exists in the pod named cnfs-nas-static-sts-1:

kubectl exec cnfs-nas-static-sts-1 -- ls -arlth /data

Expected output:

total 1G
-rw-r--r--    1 root     root        1.0G Dec 15 12:11 1G.tmpfile

The output indicates that the 1G.tmpfile file exists in the pods named cnfs-nas-static-sts-0 and cnfs-nas-static-sts-1.

Note

If you write data to be shared to different pods, CNFS cannot ensure the consistency of the data shared among the pods. Do not overwrite a shared file in case the file is damaged.

References

You can mount different directories in a NAS file system to different applications or pods to isolate volumes. For more information, see Use CNFS to manage isolated NAS volumes (recommended).
You can define automatic expansion policies for NAS volumes that are managed by CNFS to automatically expand the NAS volumes when the storage usage exceeds a specific threshold. For more information, see Use CNFS to automatically expand NAS volumes.