LVM資料卷 - Container Service for Kubernetes

您可以在本地碟上通過VolumeGroup進行磁碟虛擬化，並且通過LVM資料卷切分磁碟給應用使用。本文介紹如何使用LVM資料卷。

背景資訊

使用HostPath和LocalVolume都可以實現Pod對主機儲存空間的訪問，但均具有其局限性：

Kubernetes沒有提供上述本機存放區卷的生命週期管理能力，需要管理員手動管理、營運儲存卷。
多個Pod共用一個本機存放區時，需要共用儲存目錄或者分別使用其子目錄，無法做到容量隔離。
多個Pod共用一個本機存放區時，IOPS、吞吐等指標共用了整個儲存空間，無法進行限制。
建立Pod使用本機存放區時，不瞭解各節點儲存空間餘量，無法進行合理的儲存卷調度。

為此ACK提供了LVM資料卷方案，以解決上述問題。

功能介紹

LVM資料卷生命週期管理：卷自動建立、刪除、掛載、卸載。
LVM資料卷擴容功能。
LVM卷的監控能力。
LVM卷的IOPS限制。
節點本機存放區管理：自動營運VolumeGroup。
LVM卷的叢集容量感知能力。

注意事項

LVM本機存放區卷，不支援資料的跨節點遷移，不適合在高可用情境中使用。
VolumeGroup本機存放區容量感知為可選項（暫緩提供），VolumeGroup的初始化與本機儲存資源有關，您需要手動進行初始化或者使用自動初始化本機儲存資源進行初始化，兩者都需要您對本機儲存資源有比較深入的瞭解，否則建議您直接使用雲上的儲存資源（雲端硬碟、CNFS）等。

實現架構

基礎的LVM功能（卷的生命週期管理、擴容、掛載、格式化等）由CSI-Provisioner和CSI-Plugin實現。

單元	詳情
儲存管理器	節點本機存放區（VolumeGroup）的營運管理以及容量統計。可以不使用該組件，由Worker管理營運VolumeGroup。
儲存資訊中心	儲存節點本機存放區資訊，包括容量、VolumeGroup等資訊。
本機存放區調度器	實現建立PVC對叢集儲存容量感知功能。

步驟一：為Plugin和Provisioner組件添加Secrets的RBAC許可權

1.20叢集及以下

與csi-plugin組件共用同一個ServiceAccount，但需要先為clusterrole/alicloud-csi-plugin增加Secrets的RBAC許可權。

執行以下命令，查看叢集clusterrole/alicloud-csi-plugin是否有Secrets的create許可權。

echo `JSONPATH='{range .rules[*]}{@.resources}:{@.verbs} \r\n
{end}' \
 && kubectl get clusterrole  alicloud-csi-plugin -o jsonpath="$JSONPATH";` | grep secrets

預期輸出：

["secrets"]:["get","list"]

若clusterrole/alicloud-csi-plugin沒有Secrets的create許可權，請執行以下命令增加此許可權。

 kubectl patch clusterrole alicloud-csi-plugin --type='json' -p='[{"op": "add", "path": "/rules/0", "value":{ "apiGroups": [""], "resources": ["secrets"], "verbs": ["create"]}}]'

預期輸出：

clusterrole.rbac.authorization.k8s.io/alicloud-csi-plugin patched

執行以下命令，查看叢集中clusterrole/alicloud-csi-plugin Secrets的create許可權是否添加成功。
```
echo `JSONPATH='{range .rules[*]}{@.resources}:{@.verbs} \r\n
{end}' \
 && kubectl get clusterrole  alicloud-csi-plugin -o jsonpath="$JSONPATH";` | grep secrets
```
預期輸出：
```
["secrets"]:["create"] 
["secrets"]:["get","list"]
```
從預期輸出可得，叢集中clusterrole/alicloud-csi-plugin Secrets的create許可權已添加成功。

1.22叢集及以上

請通過下列YAML建立和授權ServiceAccount：

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: alibaba-cloud-csi-local
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims/status"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["csinodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "update", "patch", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "create", "list", "watch", "delete", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments/status"]
    verbs: ["patch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: alibaba-cloud-csi-local
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: alibaba-cloud-csi-local
subjects:
- kind: ServiceAccount
  name: alibaba-cloud-csi-local
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["csi-local-plugin-cert"]
  verbs: ["get"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["csi-plugin", "ack-cluster-profile"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: alibaba-cloud-csi-local
subjects:
- kind: ServiceAccount
  name: alibaba-cloud-csi-local
  namespace: kube-system

步驟二：部署Plugin和Provisioner組件

LVM CSI外掛程式分為2個組件：Plugin（負責掛載、卸載LVM卷）和Provisioner（負責建立LVM卷和PV對象）。

說明

請將以下YAML中的{{ regionId }}替換為您叢集所在的地區ID。

展開查看部署LVM CSI-Plugin（1.20叢集及以下）

apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
  name: localplugin.csi.alibabacloud.com
spec:
  attachRequired: false
  podInfoOnMount: true
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: csi-local-plugin
  name: csi-local-plugin
  namespace: kube-system
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: csi-local-plugin
  template:
    metadata:
      labels:
        app: csi-local-plugin
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: type
                    operator: NotIn
                    values:
                      - virtual-kubelet
      containers:
        - args:
            - '--v=5'
            - '--csi-address=/csi/csi.sock'
            - >-
              --kubelet-registration-path=/var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
          image: >-
            registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-node-driver-registrar:v1.3.0-6e9fff3-aliyun
          imagePullPolicy: Always
          name: driver-registrar
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /csi
              name: plugin-dir
            - mountPath: /registration
              name: registration-dir
        - args:
            - '--endpoint=$(CSI_ENDPOINT)'
            - '--v=5'
            - '--nodeid=$(KUBE_NODE_NAME)'
            - '--driver=localplugin.csi.alibabacloud.com'
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
            - name: SERVICE_PORT
              value: '11293'
            - name: CSI_ENDPOINT
              value: >-
                unix://var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
          image: >-
            registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
          imagePullPolicy: Always
          name: csi-localplugin
          resources: {}
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              add:
                - SYS_ADMIN
            privileged: true
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /var/lib/kubelet
              mountPropagation: Bidirectional
              name: pods-mount-dir
            - mountPath: /dev
              mountPropagation: HostToContainer
              name: host-dev
            - mountPath: /var/log/
              name: host-log
            - mountPath: /mnt
              mountPropagation: Bidirectional
              name: quota-path-dir
            - mountPath: /tls/local/grpc
              name: tls-token-dir
              readOnly: true
      dnsPolicy: ClusterFirst
      hostNetwork: true
      hostPID: true
      priorityClassName: system-node-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: csi-admin
      serviceAccountName: csi-admin
      terminationGracePeriodSeconds: 30
      tolerations:
        - operator: Exists
      volumes:
        - name: tls-token-dir
          secret:
            defaultMode: 420
            secretName: csi-local-plugin-cert
        - hostPath:
            path: /var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com
            type: DirectoryOrCreate
          name: plugin-dir
        - hostPath:
            path: /var/lib/kubelet/plugins_registry
            type: DirectoryOrCreate
          name: registration-dir
        - hostPath:
            path: /var/lib/kubelet
            type: Directory
          name: pods-mount-dir
        - hostPath:
            path: /dev
            type: ''
          name: host-dev
        - hostPath:
            path: /var/log/
            type: ''
          name: host-log
        - hostPath:
            path: /mnt
            type: Directory
          name: quota-path-dir
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 10%
    type: RollingUpdate

展開查看部署LVM CSI-Provisioner（1.20叢集及以下）

apiVersion: apps/v1
kind: Deployment
metadata:
 labels:
   app: csi-local-provisioner
 name: csi-local-provisioner
 namespace: kube-system
spec:
 selector:
   matchLabels:
     app: csi-local-provisioner
 strategy:
   rollingUpdate:
     maxSurge: 25%
     maxUnavailable: 25%
   type: RollingUpdate
 template:
   metadata:
     labels:
       app: csi-local-provisioner
   spec:
     affinity:
       nodeAffinity:
         preferredDuringSchedulingIgnoredDuringExecution:
         - preference:
             matchExpressions:
             - key: node-role.kubernetes.io/master
               operator: Exists
           weight: 1
     containers:
     - args:
       - --csi-address=$(ADDRESS)
       - --feature-gates=Topology=True
       - --volume-name-prefix=local
       - --strict-topology=true
       - --timeout=150s
       - --extra-create-metadata=true
       - --enable-leader-election=true
       - --leader-election-type=leases
       - --retry-interval-start=500ms
       - --v=5
       env:
       - name: ADDRESS
         value: /socketDir/csi.sock
       image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-provisioner:v1.6.0-71838bd-aliyun
       imagePullPolicy: Always
       name: external-local-provisioner
       volumeMounts:
       - mountPath: /socketDir
         name: socket-dir
     - name: csi-localprovisioner
       securityContext:
         privileged: true
       image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
       imagePullPolicy: "Always"
       args:
         - "--endpoint=$(CSI_ENDPOINT)"
         - "--v=2"
         - "--driver=localplugin.csi.alibabacloud.com"
       env:
       - name: CSI_ENDPOINT
         value: unix://var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com/csi.sock
       - name: SERVICE_TYPE
         value: "provisioner"
       - name: SERVICE_PORT
         value: "11290"
       volumeMounts:
       - name: socket-dir
         mountPath: /var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com
       - mountPath: /var/log/
         name: host-log
       - mountPath: /tls/local/grpc/
         name: tls-token-dir
     - args:
       - --v=5
       - --csi-address=$(ADDRESS)
       - --leader-election
       env:
       - name: ADDRESS
         value: /socketDir/csi.sock
       image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-resizer:v1.1.0-7b30758-aliyun
       imagePullPolicy: Always
       name: external-local-resizer
       volumeMounts:
       - mountPath: /socketDir/
         name: socket-dir
     hostNetwork: true
     serviceAccount: csi-admin
     tolerations:
     - effect: NoSchedule
       operator: Exists
       key: node-role.kubernetes.io/master
     - effect: NoSchedule
       operator: Exists
       key: node.cloudprovider.kubernetes.io/uninitialized
     volumes:
     - name: socket-dir
       emptyDir: {}
     - name: tls-token-dir
       emptyDir: {}
     - hostPath:
         path: /dev
         type: ""
       name: host-dev
     - hostPath:
         path: /var/log/
         type: ""
       name: host-log
     - hostPath:
         path: /mnt
         type: Directory
       name: quota-path-dir
     - hostPath:
         path: /var/lib/kubelet
         type: Directory
       name: pods-mount-dir

展開查看部署LVM CSI-Plugin（1.22叢集及以上）

apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
  name: localplugin.csi.alibabacloud.com
spec:
  attachRequired: false
  podInfoOnMount: true
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: csi-local-plugin
  name: csi-local-plugin
  namespace: kube-system
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: csi-local-plugin
  template:
    metadata:
      labels:
        app: csi-local-plugin
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: type
                    operator: NotIn
                    values:
                      - virtual-kubelet
      containers:
        - args:
            - '--v=5'
            - '--csi-address=/csi/csi.sock'
            - >-
              --kubelet-registration-path=/var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
          image: >-
            registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
          imagePullPolicy: Always
          name: driver-registrar
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /csi
              name: plugin-dir
            - mountPath: /registration
              name: registration-dir
        - args:
            - '--endpoint=$(CSI_ENDPOINT)'
            - '--v=5'
            - '--nodeid=$(KUBE_NODE_NAME)'
            - '--driver=localplugin.csi.alibabacloud.com'
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
            - name: SERVICE_PORT
              value: '11293'
            - name: CSI_ENDPOINT
              value: >-
                unix://var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
          image: >-
             registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.24.3-55228c1-aliyun
          imagePullPolicy: Always
          name: csi-localplugin
          resources: {}
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              add:
                - SYS_ADMIN
            privileged: true
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /var/lib/kubelet
              mountPropagation: Bidirectional
              name: pods-mount-dir
            - mountPath: /dev
              mountPropagation: HostToContainer
              name: host-dev
            - mountPath: /var/log/
              name: host-log
            - mountPath: /mnt
              mountPropagation: Bidirectional
              name: quota-path-dir
            - mountPath: /tls/local/grpc
              name: tls-token-dir
              readOnly: true
      dnsPolicy: ClusterFirst
      hostNetwork: true
      hostPID: true
      priorityClassName: system-node-critical
      restartPolicy: Always
      securityContext: {}
      serviceAccountName: alibaba-cloud-csi-local
      terminationGracePeriodSeconds: 30
      tolerations:
        - operator: Exists
      volumes:
        - name: tls-token-dir
          secret:
            defaultMode: 420
            secretName: csi-local-plugin-cert
        - hostPath:
            path: /var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com
            type: DirectoryOrCreate
          name: plugin-dir
        - hostPath:
            path: /var/lib/kubelet/plugins_registry
            type: DirectoryOrCreate
          name: registration-dir
        - hostPath:
            path: /var/lib/kubelet
            type: Directory
          name: pods-mount-dir
        - hostPath:
            path: /dev
            type: ''
          name: host-dev
        - hostPath:
            path: /var/log/
            type: ''
          name: host-log
        - hostPath:
            path: /mnt
            type: Directory
          name: quota-path-dir
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 10%
    type: RollingUpdate

展開查看部署LVM CSI-Provisioner（1.22叢集及以上）

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: csi-local-provisioner
  name: csi-local-provisioner
  namespace: kube-system
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: csi-local-provisioner
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: csi-local-provisioner
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - preference:
              matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
            weight: 1
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: type
                operator: NotIn
                values:
                - virtual-kubelet
      containers:
      - args:
        - --csi-address=$(ADDRESS)
        - --feature-gates=Topology=True
        - --volume-name-prefix=local
        - --strict-topology=true
        - --timeout=150s
        - --extra-create-metadata=true
        - --enable-leader-election=true
        - --leader-election-type=leases
        - --retry-interval-start=500ms
        - --default-fstype=ext4
        - --v=5
        env:
        - name: ADDRESS
          value: /socketDir/csi.sock
        image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun
        imagePullPolicy: Always
        name: external-local-provisioner
        resources: {}
        volumeMounts:
        - mountPath: /socketDir
          name: socket-dir
      - args:
        - --endpoint=$(CSI_ENDPOINT)
        - --v=2
        - --driver=localplugin.csi.alibabacloud.com
        env:
        - name: CSI_ENDPOINT
          value: unix://var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com/csi.sock
        - name: SERVICE_TYPE
          value: provisioner
        - name: SERVICE_PORT
          value: "11290"
        image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.24.3-55228c1-aliyun
        imagePullPolicy: Always
        name: csi-localprovisioner
        resources: {}
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com
          name: socket-dir
        - mountPath: /var/log/
          name: host-log
        - mountPath: /tls/local/grpc/
          name: tls-token-dir
      - args:
        - --v=5
        - --csi-address=$(ADDRESS)
        - --leader-election
        env:
        - name: ADDRESS
          value: /socketDir/csi.sock
        image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
        imagePullPolicy: Always
        name: external-local-resizer
        resources: {}
        volumeMounts:
        - mountPath: /socketDir/
          name: socket-dir
      dnsPolicy: ClusterFirst
      hostNetwork: true
      restartPolicy: Always
      securityContext: {}
      serviceAccountName: alibaba-cloud-csi-local
      terminationGracePeriodSeconds: 30
      tolerations:
      - operator: Exists
      volumes:
      - emptyDir: {}
        name: socket-dir
      - emptyDir: {}
        name: tls-token-dir
      - hostPath:
          path: /dev
          type: ""
        name: host-dev
      - hostPath:
          path: /var/log/
          type: ""
        name: host-log
      - hostPath:
          path: /mnt
          type: Directory
        name: quota-path-dir
      - hostPath:
          path: /var/lib/kubelet
          type: Directory
        name: pods-mount-dir

步驟三：使用LVM儲存卷

使用CSI-Provisioner自動建立PV，有以下特點：

StorageClass中需要指定VolumeGroup名字。
如果期望建立的PV在某個節點，需要給PVC添加Annotations：volume.kubernetes.io/selected-node: nodeName。

使用以下模板建立StorageClass。

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
    name: csi-local
provisioner: localplugin.csi.alibabacloud.com
parameters:
    volumeType: LVM
    vgName: volumegroup1
    fsType: ext4
    lvmType: "striping"
    writeIOPS: "10000"
    writeBPS: "1M"
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

參數	描述
volumeType	表示本機存放區類型為LVM（後續將支援其他類型本機存放區）。
vgName	必選，VolumeGroup的名字。
fsType	格式檔案系統類型。
lvmType	產生的LVM類型，支援linear（線性）、striping（條帶化）。
writeIOPS	使用StorageClass動態建立每一個LV時，限制每一個LV每秒寫IO的次數。
writeBPS	使用StorageClass動態建立每一個LV時，限制每一個LV每秒寫IO的大小，單位為Byte。

使用以下模板建立PVC。

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lvm-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: csi-local

使用以下模板建立應用。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-lvm
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        volumeMounts:
          - name: lvm-pvc
            mountPath: "/data"
      volumes:
        - name: lvm-pvc
          persistentVolumeClaim:
            claimName: lvm-pvc

查看應用狀態。

執行以下命令查看Pod資訊。

kubectl get pod

預期輸出：

NAME                             READY   STATUS    RESTARTS   AGE
deployment-lvm-9f798687c-m****   1/1     Running   0          9s

執行以下命令查看PVC資訊。

kubectl get pvc

預期輸出：

NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            csi-local      16s

執行以下命令查看PV資訊。

kubectl get pv

預期輸出：

NAME                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            Delete           Bound    default/lvm-pvc   csi-local               12s

執行以下命令查看Pod掛載詳情。

kubectl exec -ti deployment-lvm-9f798687c-m**** sh -- df /data

預期輸出：

Filesystem                                                                1K-blocks  Used   Available Use% Mounted on
/dev/mapper/volumegroup1-disk--afacf7a9--3d1a--45da--b443--24f8fb35****   1998672    6144   1976144   1%   /data

執行以下命令列出/data下的目錄。
```
ls /data
```
預期輸出：
```
lost+found
```
執行以下命令在/data下新增test目錄並查看。
```
touch /data/test
ls /data
```
預期輸出：
```
lost+found  test
```
執行以下命令退出登入。
```
exit
```

執行以下命令刪除Pod重建。

kubectl delete pod deployment-lvm-9f798687c-m****

預期輸出：

pod "deployment-lvm-9f798687c-m****" deleted

執行以下命令查看Pod資訊。

kubectl get pod

預期輸出：

NAME                             READY   STATUS    RESTARTS   AGE
deployment-lvm-9f798687c-j****   1/1     Running   0          2m19s

執行以下命令查看Pod掛載詳情。

kubectl exec deployment-lvm-9f798687c-j**** -- ls /data

預期輸出：

lost+found  
test

擴容LVM卷。

執行以下命令查看PVC資訊。

kubectl get pvc

預期輸出：

NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            csi-local      6m50s

執行以下命令將PVC擴容到4 GiB。

kubectl patch pvc lvm-pvc -p '{"spec":{"resources":{"requests":{"storage":"4Gi"}}}}'

預期輸出：

persistentvolumeclaim/lvm-pvc patched

執行以下命令查看PVC資訊。

kubectl get pvc

預期輸出：

NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   4Gi        RWO            csi-local      7m26s

執行以下命令查看LVM卷是否擴容到4 GiB。

kubectl exec deployment-lvm-9f798687c-j**** -- df /data

預期輸出：

Filesystem                                                                1K-blocks  Used   Available Use% Mounted on
/dev/mapper/volumegroup1-disk--afacf7a9--3d1a--45da--b443--24f8fb35****   4062912    8184   4038344   1%   /data

執行以下命令監控LVM卷。

curl -s localhost:10255/metrics | grep lvm-pvc

預期輸出：

kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 1.917165568e+09
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 1.939816448e+09
kubelet_volume_stats_inodes{namespace="default",persistentvolumeclaim="lvm-pvc"} 122400
kubelet_volume_stats_inodes_free{namespace="default",persistentvolumeclaim="lvm-pvc"} 122389
kubelet_volume_stats_inodes_used{namespace="default",persistentvolumeclaim="lvm-pvc"} 11
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 5.873664e+06

可以將上述監控資料接入Prometheus並在前端展示。具體操作，請參見開源Prometheus監控。