How to restore data in an ESSD by using an IA snapshot - Container Service for Kubernetes

If data loss occurs due to an accidental ESSD deletion, you can use the instant access (IA) snapshot of the ESSD to restore the data in the ESSD. This topic describes how to restore data in an ESSD by using an IA snapshot.

Prerequisites

A Container Service for Kubernetes (ACK) that runs Kubernetes 1.20 or later is created. For more information, see Create an ACK managed cluster.
Log on the Elastic Compute Service (ECS) console and select the region where your cluster is deployed. Make sure that the ECS Snapshot service is enabled. For more information, see Activate ECS Snapshot.
The versions of csi-plugin and csi-provisioner are 1.24.4-7371f039-aliyun or later. For more information about how to update csi-plugin and csi-provisioner, see Manage the CSI plug-in.

Scenarios

Data loss occurs due to accidental ESSD deletions

If the reclaim policy of an ESSD is Delete, after you delete the persistent volume claim (PVC) that is used to mount the ESSD, the related persistent volume (PV) and ESSD are also deleted. When you accidentally deleted a disk, you can use a snapshot of the disk to restore the disk data to the point in time when the snapshot was created. You cannot restore disk data if no snapshot is created. You cannot restore the data that is written into the disk after the snapshot was created.

To ensure that you can restore disk data when you accidentally delete an ESSD, ACK can automatically create IA snapshots for ESSDs before disk deletion. The IA snapshots created by ACK are retained for a specific period of time. This allows you to restore the data in ESSDs by using the IA snapshots created by ACK.

Benefits

If a disk is accidentally deleted, you can restore the disk data by using an IA snapshot of the disk.
The IA snapshot feature does not affect the deletion operation because IA snapshots can be created within seconds.
Note
Only ESSDs support IA snapshots.

Configurations

csi-provisioner configurations

Run the following command to add the following environment variable to the containers parameter of csi-provisioner in the kube-system namespace: VOLUME_DEL_AUTO_SNAP: "true".

kubectl patch deploy csi-provisioner -n kube-system -p '{"spec":{"template":{"spec":{"containers":[{"name":"csi-provisioner","env":[{"name":"VOLUME_DEL_AUTO_SNAP","value":"true"}]}]}}}}'

Note

ACK can automatically create IA snapshots for ESSDs before disk deletion only when you set VOLUME_DEL_AUTO_SNAP to true.

StorageClass configurations

Create a StorageClass based on the following template:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-datasafe-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
  type: cloud_essd
  volumeDeleteSnapshotRetentionDays: "3"   # This setting takes effect only when the reclaimPolicy parameter is set to "Delete". 
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
allowVolumeExpansion: true

Parameter

Description

volumeDeleteSnapshotRetentionDays

The number of days for which the system retains the IA snapshot that is automatically created before the ESSD is expanded.

If you leave this parameter empty, the system cannot create IA snapshots. We recommend that you set this parameter.

Create an IA snapshot

In this section, a MySQL application is created and has an ESSD mounted.

Create a file named mysql.yaml based on the following requirements:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: essd-pvc
  namespace: autosnapshot
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 25Gi
  storageClassName: alicloud-datasafe-essd  # Use the StorageClass that you created in the previous section. 
---
apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
  namespace: autosnapshot
type: Opaque
data:
  username: dGVzdDEK
  password: dGVzdDEtdmFsdWUK
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql-sts
  namespace: autosnapshot
spec:
  selector:
    matchLabels:
      app: mysql-sts
  serviceName: mysql-sts
  template:
    metadata:
      labels:
        app: mysql-sts
    spec:
      containers:
      - name: mysql-sts
        image: mysql:5.7
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        ports:
        - containerPort: 80
          name: mysql-sts
        volumeMounts:
        - name: mysql
          mountPath: /var/lib/mysql
          subPath: mysql
      volumes:
        - name: mysql
          persistentVolumeClaim:
            claimName: essd-pvc

Run the following command to deploy the MySQL application:
```
kubectl apply -f mysql.yaml
```
Run the following command to query the disk volume that is mounted to the MySQL application:
```
kubectl get pvc -nautosnapshot | grep essd-pvc
```
Expected output:
```
essd-pvc   Bound    d-2zeit7uza22vjya1****   25Gi       RWO            alicloud-datasafe-essd   54s
```
The ID of the mounted ESSD is d-2zeit7uza22vjya1****.
Run the following command to access the container of the MySQL application:
```
kubectl -n autosnapshot exec -it mysql-sts-0 -- /bin/sh
```
Run the following command in a shell of the container to write data into the container:
```
dd if=/dev/urandom of=/var/lib/mysql/mysql/record.txt bs=1M count=1000
```
Run the following command to query the size of the data that is written into the container:
```
ls /var/lib/mysql/mysql -l | grep record
```
Expected output:
```
-rw-r--r-- 1 root  root  1048576000 Nov  8 02:36 record.txt
```

Delete the disk

Run the following command to reduce the number of replicated pods to 0 for the MySQL application:
```
kubectl scale sts/mysql-sts -n autosnapshot --replicas=0
```
Run the following command to delete the PVC that is used by the MySQL application:
```
kubectl delete pvc essd-pvc -n autosnapshot
```
Run the following command to query the VolumeSnapshot that is created for the IA snapshot:
```
kubectl get volumesnapshot
```
Expected output:
```
d-2zeit7uza22vjya1****-delprotect                   true                                d-2zeit7uza22vjya1****-delprotect-content   30Gi                               d-2zeit7uza22vjya1****-delprotect-content          6s             6s
```
The following list describes some parameters in the preceding output:
- d-2zeit7uza22vjya1****: the ID of the ESSD that is deleted. The ID is the same as the ID of the ESSD that is mounted by using the essd-pvc PVC.
- d-2zeit7uza22vjya1****-delprotect: the name of the VolumeSnapshot that is created for the IA snapshot.
- d-2zeit7uza22vjya1****-delprotect-content: the name of the VolumeSnapshotContent that is created for the IA snapshot.

Use the IA snapshot to restore data after the disk is deleted

In this section, the IA snapshot created in the previous Delete the disk section is used to restore the disk data after the disk is deleted.

Note

By default, the original VolumeSnapshot created for the IA snapshot belongs to the default namespace. If your application is deployed in a different namespace, you must create a VolumeSnapshot in the namespace for the IA snapshot. In this example, the MySQL application is deployed in the autosnapshot namespace. You must create a VolumeSnapshot in the autosnapshot namespace.

Run the following command to query the value of the snapshotHandle parameter of the VolumeSnapshotContent that is created for the IA snapshot:
```
kubectl get volumesnapshotcontent d-2zeit7uza22vjya1****-delprotect-content -oyaml | grep snapshotHandle
```
Expected output:
```
snapshotHandle: s-2zegw6gmuc866xgc****
```

Create a VolumeSnapshotContent by using the following YAML template. The template specifies the VolumeSnapshot to be created.

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
  name: datasafe-volumesnapshotcontent
spec:
  deletionPolicy: Retain
  driver: diskplugin.csi.alibabacloud.com
  source:
    snapshotHandle: s-2zegw6gmuc866xgc****   # Set to the value of the snapshotHandle parameter of the original VolumeSnapshotContent. 
  volumeSnapshotRef:
    name: datasafe-volumesnapshot       # The name of the VolumeSnapshot to be created. 
    namespace: autosnapshot             # The namespace in which the MySQL application is deployed.

Create a VolumeSnapshot in the namespace in which the MySQL application is deployed by using the following YAML template:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: datasafe-volumesnapshot
  namespace: autosnapshot
spec:
  source:
    volumeSnapshotContentName: datasafe-volumesnapshotcontent

Create a PVC by using the following YAML template. The template specifies the VolumeSnapshot to be used.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: essd-pvc       # Set to the name of the PVC used by the MySQL application. 
  namespace: autosnapshot
spec:
  accessModes: [ "ReadWriteOnce" ]
  storageClassName: alicloud-datasafe-essd
  resources:
    requests:
      storage: 30Gi
  dataSource:
    name: datasafe-volumesnapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io

Run the following command to increase the number of replicated pods to 1 for the MySQL application:
```
kubectl scale sts/mysql-sts -n autosnapshot --replicas=1
```
Run the following command to check whether the PVC is mounted to the MySQL application:
```
kubectl describe pvc essd-pvc -n autosnapshot | grep "Used By"
```
Expected output:
```
Used By:     mysql-sts-0
```
Run the following command to access the container of the MySQL application:
```
kubectl -n autosnapshot exec -it mysql-sts-0 -- /bin/sh
```
Run the following command to check whether the data that you previously wrote into the container is restored:
```
ls /var/lib/mysql/mysql -l | grep record
```
Expected output:
```
-rw-r--r-- 1 root  root  1048576000 Nov  8 02:36 record.txt
```
In the output, 1048576000 is returned for the size of the data. The size is the same as the size of the data that you previously wrote into the container. This indicates that the data restoration is successful.