The csi-compatible-controller component allows you to install and use Container Storage Interface (CSI) in a Container Service for Kubernetes (ACK) cluster that uses FlexVolume. You can install and use this component to progressively convert volumes managed by FlexVolume into volumes managed by CSI. This topic describes how to use the csi-compatible-controller component to migrate from FlexVolume to CSI.
Table of contents
Prepare the environment
Install the csi-compatible-controller component
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Operations > Add-ons.
On the Add-ons page, click the Storage tab, find csi-compatible-controller, click Install in the lower-right part of the card, and then click OK.
Install CSI
After you install csi-compatible-controller, use the following YAML files to install CSI. You cannot install CSI from the Add-ons page in the ACK console because the CSI configurations vary based on the Kubernetes version of the ACK cluster. Select the following YAML files based on the Kubernetes version of the ACK cluster. Run the kubectl apply command to install the YAML files in the ACK cluster.
Note
Replace the region ID of the image in the YAML files. For example, replace
cn-hangzhouwith the region ID of the ACK cluster.Replace
{{.ClusterID}}in the following YAML files with the ID of your ACK cluster.
ACK cluster that runs Kubernetes versions 1.20 or later
YAML file of csi-provisioner
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/control-plane
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
hostPID: true
priorityClassName: system-node-critical
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--extra-create-metadata=true"
- "--default-fstype=ext4"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-attacher:v3.3-72dd428b-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--default-fstype=nfs"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-oss-provisioner
args:
- --csi-address=$(ADDRESS)
- --volume-name-prefix=oss
- --timeout=150s
- --leader-election=true
- --retry-interval-start=500ms
- --default-fstype=ossfs
- --v=5
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com/csi.sock
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 10m
memory: 16Mi
volumeMounts:
- mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
name: oss-provisioner-dir
- name: external-csi-snapshotter
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-snapshotter:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/snapshot-controller:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--leader-election=true"
- name: csi-provisioner
securityContext:
privileged: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.8-e724570-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk,oss"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-log
mountPath: /var/log/
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: oss-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /mnt
mountPropagation: Bidirectional
name: host-mnt
- mountPath: /host/etc
name: etc
resources:
limits:
cpu: 500m
memory: 1024Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: oss-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
type: ""
- name: host-mnt
hostPath:
path: /mnt
type: ""
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
YAML file of csi-plugin
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: kube-system
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-ossfs-credentials"]
verbs: ["get", "patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch", "create", "delete" ]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-local-plugin-cert"]
verbs: ["get"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
- apiGroups: [""]
resources: ["endpoints"]
resourceNames: ["cnfs-cache-ds-service"]
verbs: ["get"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["csi-plugin", "ack-cluster-profile"]
verbs: ["get"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["storage-monitor-service"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "update", "patch", "list", "watch"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch", "create"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update", "create"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
- apiGroups: [""]
resources: ["nodes/stats"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: alicloud-csi-plugin
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: alicloud-csi-plugin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostIPC: true
hostPID: true
dnsPolicy: ClusterFirst
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
allowPrivilegeEscalation: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.8-e724570-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: DEFAULT_REGISTRY
value: registry-vpc.cn-shenzhen.aliyuncs.com
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: SERVICE_TYPE
value: "plugin"
- name: MAX_VOLUMES_PERNODE
value: "15"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /host/var/run/ossfs
name: ossfs-metrics-dir
- mountPath: /host/var/run/efc
name: efc-metrics-dir
- mountPath: /etc/csi-plugin/config
name: csi-plugin-cm
- name: host-mnt
mountPath: /mnt
mountPropagation: "Bidirectional"
- mountPath: /run/kata-containers/shared/direct-volumes
name: kata-direct-volumes
volumes:
- name: ossfs-metrics-dir
hostPath:
path: /var/run/ossfs
type: DirectoryOrCreate
- name: efc-metrics-dir
hostPath:
path: /var/run/efc
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: host-mnt
hostPath:
path: /mnt
type: DirectoryOrCreate
- name: csi-plugin-cm
configMap:
name: csi-plugin
optional: true
- name: kata-direct-volumes
hostPath:
path: /run/kata-containers/shared/direct-volumes
type: DirectoryOrCreate
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 20%
type: RollingUpdate
ACK cluster that runs Kubernetes versions earlier than 1.20
YAML file of csi-provisioner
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=diskplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-attacher:v2.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=nasplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-csi-snapshotter
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-snapshotter:v4.0.0-1f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
imagePullPolicy: Always
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/snapshot-controller:v4.0.0-41f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--leader-election=true"
imagePullPolicy: Always
- name: csi-provisioner
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- name: host-log
mountPath: /var/log/
- name: etc
mountPath: /host/etc
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: host-dev
hostPath:
path: /dev
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
- name: etc
hostPath:
path: /etcYAML file of csi-plugin
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "list"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "watch", "delete", "get", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods","pods/exec"]
verbs: ["create", "delete", "get", "post", "list", "watch", "patch", "udpate"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: alicloud-csi-plugin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
beta.kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostPID: true
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.18.8.51-c504ef45-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "plugin"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
volumes:
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 10%
type: RollingUpdateVerify CSI
Create a StatefulSet and check the status of the pods created by the StatefulSet. If the status of the pods is running, CSI is deployed and the environment is ready for volume conversion.
YAML file of the StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-csi-tr
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: disk-csi-tr
mountPath: /data
volumeClaimTemplates:
- metadata:
name: disk-csi-tr
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "csi-alicloud-disk-topology-alltype"
resources:
requests:
storage: 20Gi
After the preceding components are installed, you can use FlexVolume and CSI in the same cluster. You can convert volumes managed by FlexVolume to volumes managed by CSI.
Configure an application to use volumes managed by CSI
You need to convert all volumes managed by FlexVolume into volumes managed by CSI. After the conversion is complete, you can uninstall FlexVolume.
Step 1: Convert PVCs and PVs managed by FlexVolume into PVCs and PVs managed by CSI
Query all persistent volume claims (PVCs) and persistent volumes (PVs) managed by FlexVolume and convert them into PVCs and PVs managed by CSI. You can use the Flexvolume2CSI CLI to convert multiple PVCs and PVs at the same time. For more information, see Use Flexvolume2CSI to batch convert PVs and PVCs.
In this topic, disk volumes are used as an example to demonstrate how to convert PVs and PVCs managed by FlexVolume into PVs and PVCs managed by CSI. You can modify the CSI template based on your business requirements.
PVC and PV managed by FlexVolume
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: Delete
storageClassName: alicloud-disk-available
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
accessModes:
- ReadWriteOnce
storageClassName: alicloud-disk-available
resources:
requests:
storage: 20GiPVC and PV managed by CSI
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
csi:
driver: diskplugin.csi.alibabacloud.com
volumeHandle: d-bp1bnp9homa0tyv6****
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
volumeName: d-bp1bnp9homa0tyv6****Step 2: Change the reclaim policy
To avoid volume loss, you need to check the PVC and PV managed by FlexVolume before conversion. If the pv.Spec.persistentVolumeReclaimPolicy parameter is set to Delete, change the value to Retain.
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: **Retain**
storageClassName: alicloud-disk-available
volumeMode: FilesystemStep 3: Change the type of volume used by an application
Perform the following steps to change the type of volume used by an application from FlexVolume to CSI.
Run the following command to scale the number of replicated pods created by the StatefulSet to 0:
kubectl scale --replicas=0 sts/sts-testRun the following command to delete the PVC and PV managed by FlexVolume:
kubectl delete pvc pvc-disk kubectl delete pv d-bp1bnp9homa0tyv6****Run the following command to create a PVC and a PV managed by CSI:
kuectl apply -f csi-pvc-pv.yamlRun the following command to scale the number of replicated pods created by the StatefulSet to 3:
kubectl scale --replicas=3 sts/sts-testNoteYou can follow the same procedure to convert volumes created by using pvcClaimName or volumes that are used by Deployments.
You can repeat the same steps to change the type of volume used by other applications in the cluster. After you modify all applications, you can uninstall FlexVolume. For more information, see Upgrade from FlexVolume to CSI for clusters where no data is stored.
Manage components
Update components
After you perform the preceding steps to install CSI in an ACK cluster that uses FlexVolume, you can run the following commands to update CSI to the latest version. You can obtain the updated YAML files in the preceding section and run the kubectl apply command to install the YAML files in the cluster.
If you want to update CSI on the Add-ons page in the ACK console, you need to first convert all PVs and PVCs managed by FlexVolume into PVs and PVCs managed by CSI. You cannot update CSI om the Add-ons page when both FlexVolume and CSI are used in the cluster. For more information, see Upgrade from FlexVolume to CSI for clusters where no data is stored.
Delete components
After you migrate from FlexVolume to CSI, you can delete csi-compatible-controller if the following conditions are met:
The FlexVolume component is deleted.
All PVs managed by FlexVolume are deleted.
The
--enable-controller-attach-detachparameter of the kubelet is set totrue. If this condition is not met, set the parameter to true. For more information, see Step 3 and Step 4 in Upgrade from FlexVolume to CSI for clusters where no data is stored.
Reference
For more information about how to update from FlexVolume to CSI for clusters in which no data is stored, see Upgrade from FlexVolume to CSI for clusters where no data is stored.