このトピックでは、KubernetesクラスターにContainer Storage Interface (CSI) プラグインを手動でインストールし、Apsara File Storage NAS (NAS) ファイルシステムをKubernetesクラスターにマウントする方法について説明します。
CSIプラグインは、オープンソースコンポーネントである。 Alibaba Cloudは、CSIプラグインのバージョン、セキュリティ、および精度を保証するものではありません。 Alibaba Cloudは、CSIプラグインの使用中に発生したいかなる損害についても責任を負いません。 ビジネス要件に基づいて、適切な方法を選択してNASファイルシステムをマウントすることを推奨します。 CSIプラグインを使用しているときに問題が発生した場合は、GitHubで問題を送信できます。
前提条件
ネットワークファイルシステム (NFS) ファイルシステムが作成されます。 詳細については、「ファイルシステムの作成」をご参照ください。
NASファイルシステムのデータを暗号化するには、NASファイルシステムの作成時に暗号化タイプを設定する必要があります。
マウントターゲットが作成されます。 詳細については、「マウントターゲットの管理」をご参照ください。
自己管理型Kubernetesクラスターが作成され、クラスターバージョンが1.16以降になります。
ポート111とポート2049は、ポッドスケジューリングノードのセキュリティグループルールで指定されています。 詳細については、「セキュリティグループルールの追加」をご参照ください。
自己管理型Kubernetesクラスターがデータセンターにデプロイされている場合、データセンターはAlibaba Cloudに接続されている必要があります。 Express connect、Smart Access Gateway (SAG) 、またはVPN Gatewayを使用して、データセンターをAlibaba Cloudに接続できます。 詳細については、以下のトピックをご参照ください。
制限事項
マウントシナリオの制限、ファイルシステムの数、およびファイル共有プロトコルは、NASファイルシステムのストレージクラスによって異なります。 詳細については、「制限事項」をご参照ください。
NASファイルシステムをアンマウントする前に、ファイルシステムのマウント対象を削除しないでください。 そうしないと、オペレーティングシステムのハングが発生する可能性があります。
手順
CSIプラグインをデプロイします。
CSI-ALLのデプロイに使用するYAMLファイルを作成します。
csi-all.yamlという名前のファイルを作成し、次のテンプレートをファイルにコピーします。 ACCESS_KEY_IDおよびACCESS_KEY_SECRETパラメーターの値を、Alibaba CloudアカウントのAccessKey IDおよびAccessKey secretに置き換えます。 Alibaba CloudアカウントのAccessKeyペアを取得する方法の詳細については、「AccessKeyペアの作成」をご参照ください。
csi-all.yaml
--- apiVersion: v1 kind: ServiceAccount metadata: name: csi-admin namespace: kube-system --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: alicloud-csi-plugin rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "create", "list"] - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "update", "create", "delete", "patch"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: [""] resources: ["persistentvolumeclaims/status"] verbs: ["get", "list", "watch", "update", "patch"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: ["storage.k8s.io"] resources: ["csinodes"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["get", "list", "watch", "create", "update", "patch"] - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "watch", "list", "delete", "update", "create"] - apiGroups: [""] resources: ["configmaps"] verbs: ["get", "watch", "list", "delete", "update", "create"] - apiGroups: [""] resources: ["nodes"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["csi.storage.k8s.io"] resources: ["csinodeinfos"] verbs: ["get", "list", "watch"] - apiGroups: ["storage.k8s.io"] resources: ["volumeattachments"] verbs: ["get", "list", "watch", "update", "patch"] - apiGroups: ["snapshot.storage.k8s.io"] resources: ["volumesnapshotclasses"] verbs: ["get", "list", "watch", "create"] - apiGroups: ["snapshot.storage.k8s.io"] resources: ["volumesnapshotcontents"] verbs: ["create", "get", "list", "watch", "update", "delete"] - apiGroups: ["snapshot.storage.k8s.io"] resources: ["volumesnapshots"] verbs: ["get", "list", "watch", "update", "create"] - apiGroups: ["apiextensions.k8s.io"] resources: ["customresourcedefinitions"] verbs: ["create", "list", "watch", "delete", "get", "update", "patch"] - apiGroups: ["coordination.k8s.io"] resources: ["leases"] verbs: ["get", "create", "list", "watch", "delete", "update"] - apiGroups: ["snapshot.storage.k8s.io"] resources: ["volumesnapshotcontents/status"] verbs: ["update"] - apiGroups: ["storage.k8s.io"] resources: ["volumeattachments/status"] verbs: ["patch"] - apiGroups: ["snapshot.storage.k8s.io"] resources: ["volumesnapshots/status"] verbs: ["update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["namespaces"] verbs: ["get", "list"] - apiGroups: [""] resources: ["pods","pods/exec"] verbs: ["create", "delete", "get", "post", "list", "watch", "patch", "udpate"] - apiGroups: ["storage.alibabacloud.com"] resources: ["rules"] verbs: ["get"] - apiGroups: ["storage.alibabacloud.com"] resources: ["containernetworkfilesystems"] verbs: ["get","list", "watch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: alicloud-csi-plugin subjects: - kind: ServiceAccount name: csi-admin namespace: kube-system roleRef: kind: ClusterRole name: alicloud-csi-plugin apiGroup: rbac.authorization.k8s.io --- apiVersion: storage.k8s.io/v1 kind: CSIDriver metadata: name: diskplugin.csi.alibabacloud.com spec: attachRequired: true podInfoOnMount: true --- apiVersion: storage.k8s.io/v1 kind: CSIDriver metadata: name: nasplugin.csi.alibabacloud.com spec: attachRequired: false podInfoOnMount: true --- apiVersion: storage.k8s.io/v1 kind: CSIDriver metadata: name: ossplugin.csi.alibabacloud.com spec: attachRequired: false podInfoOnMount: true --- kind: DaemonSet apiVersion: apps/v1 metadata: name: csi-plugin namespace: kube-system spec: selector: matchLabels: app: csi-plugin template: metadata: labels: app: csi-plugin spec: tolerations: - operator: Exists affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: type operator: NotIn values: - virtual-kubelet nodeSelector: kubernetes.io/os: linux serviceAccount: csi-admin priorityClassName: system-node-critical hostNetwork: true hostPID: true dnsPolicy: ClusterFirst containers: - name: disk-driver-registrar image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock" - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock" volumeMounts: - name: kubelet-dir mountPath: /var/lib/kubelet - name: registration-dir mountPath: /registration - name: nas-driver-registrar image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock" - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock" volumeMounts: - name: kubelet-dir mountPath: /var/lib/kubelet/ - name: registration-dir mountPath: /registration - name: oss-driver-registrar image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock" - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock" volumeMounts: - name: kubelet-dir mountPath: /var/lib/kubelet/ - name: registration-dir mountPath: /registration - name: csi-plugin securityContext: privileged: true allowPrivilegeEscalation: true image: registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.24.5-39a3970-aliyun args: - "--endpoint=$(CSI_ENDPOINT)" - "--v=2" - "--driver=oss,nas,disk" env: - name: KUBE_NODE_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: spec.nodeName - name: CSI_ENDPOINT value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock - name: MAX_VOLUMES_PERNODE value: "15" - name: SERVICE_TYPE value: "plugin" - name: ACCESS_KEY_ID value: "xxx" - name: ACCESS_KEY_SECRET value: "xxx" - name: KUBELET_ROOT_DIR value: "/var/lib/kubelet" resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 1024Mi livenessProbe: httpGet: path: /healthz port: healthz scheme: HTTP initialDelaySeconds: 10 periodSeconds: 30 timeoutSeconds: 5 failureThreshold: 5 readinessProbe: httpGet: path: /healthz port: healthz initialDelaySeconds: 10 periodSeconds: 30 timeoutSeconds: 5 failureThreshold: 5 ports: - name: healthz containerPort: 11260 volumeMounts: - name: kubelet-dir mountPath: /var/lib/kubelet/ mountPropagation: "Bidirectional" - name: etc mountPath: /host/etc - name: host-log mountPath: /var/log/ - name: ossconnectordir mountPath: /host/usr/ - name: container-dir mountPath: /var/lib/container mountPropagation: "Bidirectional" - name: host-dev mountPath: /dev mountPropagation: "HostToContainer" - mountPath: /host/var/run/ name: fuse-metrics-dir volumes: - name: fuse-metrics-dir hostPath: path: /var/run/ type: DirectoryOrCreate - name: registration-dir hostPath: path: /var/lib/kubelet/plugins_registry type: DirectoryOrCreate - name: container-dir hostPath: path: /var/lib/container type: DirectoryOrCreate - name: kubelet-dir hostPath: path: /var/lib/kubelet type: Directory - name: host-dev hostPath: path: /dev - name: host-log hostPath: path: /var/log/ - name: etc hostPath: path: /etc - name: ossconnectordir hostPath: path: /usr/ updateStrategy: rollingUpdate: maxUnavailable: 30% type: RollingUpdate --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-topology-alltype parameters: type: cloud_essd,cloud_ssd,cloud_efficiency provisioner: diskplugin.csi.alibabacloud.com reclaimPolicy: Delete allowVolumeExpansion: true volumeBindingMode: WaitForFirstConsumer --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-available provisioner: diskplugin.csi.alibabacloud.com parameters: type: available reclaimPolicy: Delete allowVolumeExpansion: true --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-essd provisioner: diskplugin.csi.alibabacloud.com parameters: type: cloud_essd reclaimPolicy: Delete allowVolumeExpansion: true --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-ssd provisioner: diskplugin.csi.alibabacloud.com parameters: type: cloud_ssd reclaimPolicy: Delete allowVolumeExpansion: true --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-efficiency provisioner: diskplugin.csi.alibabacloud.com parameters: type: cloud_efficiency reclaimPolicy: Delete allowVolumeExpansion: true --- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-topology provisioner: diskplugin.csi.alibabacloud.com parameters: type: available reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true --- kind: Deployment apiVersion: apps/v1 metadata: name: csi-provisioner namespace: kube-system spec: selector: matchLabels: app: csi-provisioner strategy: rollingUpdate: maxSurge: 0 maxUnavailable: 1 type: RollingUpdate replicas: 2 template: metadata: labels: app: csi-provisioner spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: node-role.kubernetes.io/master operator: Exists requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: type operator: NotIn values: - virtual-kubelet podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: app operator: In values: - csi-provisioner topologyKey: kubernetes.io/hostname tolerations: - effect: NoSchedule operator: Exists key: node-role.kubernetes.io/master - effect: NoSchedule operator: Exists key: node.cloudprovider.kubernetes.io/uninitialized serviceAccount: csi-admin hostPID: true priorityClassName: system-node-critical containers: - name: external-disk-provisioner image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--csi-address=$(ADDRESS)" - "--feature-gates=Topology=True" - "--volume-name-prefix=disk" - "--strict-topology=true" - "--timeout=150s" - "--leader-election=true" - "--retry-interval-start=500ms" - "--extra-create-metadata=true" - "--default-fstype=ext4" - "--v=5" env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock volumeMounts: - name: disk-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com - name: external-disk-attacher image: registry.cn-beijing.aliyuncs.com/acs/csi-attacher:v3.3-72dd428b-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=$(ADDRESS)" - "--leader-election=true" env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock volumeMounts: - name: disk-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com - name: external-disk-resizer image: registry.cn-beijing.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=$(ADDRESS)" - "--leader-election" env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock volumeMounts: - name: disk-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com - name: external-nas-provisioner image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--csi-address=$(ADDRESS)" - "--volume-name-prefix=nas" - "--timeout=150s" - "--leader-election=true" - "--retry-interval-start=500ms" - "--default-fstype=nfs" - "--v=5" env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock volumeMounts: - name: nas-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com - name: external-nas-resizer image: registry.cn-beijing.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=$(ADDRESS)" - "--leader-election" env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock volumeMounts: - name: nas-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com - name: external-oss-provisioner args: - --csi-address=$(ADDRESS) - --volume-name-prefix=oss - --timeout=150s - --leader-election=true - --retry-interval-start=500ms - --default-fstype=ossfs - --v=5 env: - name: ADDRESS value: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com/csi.sock image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun resources: limits: cpu: 500m memory: 1Gi requests: cpu: 10m memory: 16Mi volumeMounts: - mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com name: oss-provisioner-dir - name: external-csi-snapshotter image: registry.cn-beijing.aliyuncs.com/acs/csi-snapshotter:v4.0.0-a230d5b3-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--csi-address=$(ADDRESS)" - "--leader-election=true" - "--extra-create-metadata=true" env: - name: ADDRESS value: /csi/csi.sock volumeMounts: - name: disk-provisioner-dir mountPath: /csi - name: external-snapshot-controller image: registry.cn-beijing.aliyuncs.com/acs/snapshot-controller:v4.0.0-a230d5b3-aliyun resources: requests: cpu: 10m memory: 16Mi limits: cpu: 500m memory: 1024Mi args: - "--v=5" - "--leader-election=true" - name: csi-provisioner securityContext: privileged: true image: registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.24.5-39a3970-aliyun args: - "--endpoint=$(CSI_ENDPOINT)" - "--v=2" - "--driver=nas,disk,oss" env: - name: CSI_ENDPOINT value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock - name: MAX_VOLUMES_PERNODE value: "15" - name: SERVICE_TYPE value: "provisioner" - name: "CLUSTER_ID" value: "abc" - name: "ACCESS_KEY_ID" value: "xxx" - name: "ACCESS_KEY_SECRET" value: "xxx" livenessProbe: httpGet: path: /healthz port: healthz scheme: HTTP initialDelaySeconds: 10 periodSeconds: 30 timeoutSeconds: 5 failureThreshold: 5 readinessProbe: httpGet: path: /healthz port: healthz initialDelaySeconds: 5 periodSeconds: 20 ports: - name: healthz containerPort: 11270 volumeMounts: - name: host-log mountPath: /var/log/ - name: disk-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com - name: nas-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com - name: oss-provisioner-dir mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com - mountPath: /var/addon name: addon-token readOnly: true - mountPath: /mnt mountPropagation: Bidirectional name: host-dev - mountPath: /host/etc name: etc resources: limits: cpu: 500m memory: 1024Mi requests: cpu: 100m memory: 128Mi volumes: - name: disk-provisioner-dir emptyDir: {} - name: nas-provisioner-dir emptyDir: {} - name: oss-provisioner-dir emptyDir: {} - name: host-log hostPath: path: /var/log/ - name: etc hostPath: path: /etc type: "" - name: host-dev hostPath: path: /mnt type: "" - name: addon-token secret: defaultMode: 420 optional: true items: - key: addon.token.config path: token-config secretName: addon.csi.token
CSI-ALLをデプロイします。
kubectl apply -f csi-all.yaml
デプロイの結果を表示します。
kubectl -n kube-system get pods
期待されるコマンド出力:
NAME READY STATUS RESTARTS AGE csi-plugin-5h6gq 4/4 Running 0 30m csi-plugin-gjjp7 4/4 Running 0 31m csi-provisioner-cfc45d84c-lzpvm 9/9 Running 0 31m csi-provisioner-cfc45d84c-zczfn 9/9 Running 0 31m ......
ファイルシステムをマウントします。
静的にプロビジョニングされたNASボリュームをマウントする方法の詳細については、「NFSを使用して静的にプロビジョニングされたNASボリュームをマウントする」をご参照ください。
動的にプロビジョニングされたNASボリュームをマウントする方法の詳細については、「NFSを使用して動的にプロビジョニングされたNASボリュームをマウントする」をご参照ください。