Use Terraform to migrate applications across clusters - Container Service for Kubernetes

You can use the Terraform command line to deploy custom resources (CRs) for backup repositories, application backups, and application restores in your clusters to migrate applications across them.

Prerequisites

An ACK managed cluster that runs Kubernetes 1.18 or later has been created using Terraform. For more information, see Use Terraform to create an ACK managed cluster.
The migrate-controller component has been installed and the required permissions have been granted. For more information, see Install the migrate-controller component and grant permissions. You can also use Terraform to install components. For more information, see Use Terraform to install components.
An Object Storage Service (OSS) bucket has been created to store backups. The bucket name must start with cnfs-oss-. For more information, see Create a bucket using Terraform and alicloud_oss_bucket.
If you want to back up persistent volumes, you must activate the ECS Snapshot service and the Cloud Backup service. For more information, see Activate OSS, Activate ECS Snapshot, and Activate Cloud Backup.

Considerations

Do not use the kubectl delete command to delete backup or restore jobs. Otherwise, the related cloud resources may not be completely deleted. To delete backup-related resources, see Step 5: Delete resources from the backup center.
Pay close attention to the new releases of the backup center component named migrate-controller and update the component to the latest version at your earliest convenience. For more information, see Manage components.
Do not delete the parameters in the following sample code in case the backup files cannot be restored.

Step 1: Associate a cluster

You can use the Kubernetes provider to associate a cluster. For more information, see Kubernetes Provider.

This example shows how to associate a cluster using a KubeConfig file.

Create a Terraform working directory.
In the Terraform working directory, create a configuration file named csdr.tf. Add the following content to the file to configure the provider to use your KubeConfig file.
```
provider "kubernetes" {
  config_path = "~/.kube/config"
}
```

Run the following command to initialize the Terraform runtime environment.

terraform init

If the following output is returned, Terraform has been initialized.

View output

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/alicloud from the dependency lock file
- Reusing previous version of hashicorp/kubernetes from the dependency lock file
- Using previously-installed hashicorp/alicloud v1.223.1
- Using previously-installed hashicorp/kubernetes v2.30.0

╷
│ Warning: Additional provider information from registry
│
│ The remote registry returned warnings for registry.terraform.io/hashicorp/alicloud:
│ - For users on Terraform 0.13 or greater, this provider has moved to aliyun/alicloud. Please
│ update your source in required_providers.
╵

Terraform has been successfully initialized!

Step 2: Create a backup repository

Add the following content to the csdr.tf configuration file and modify the content as needed.

resource "kubernetes_manifest" "backuplocation-demo" {
  manifest = {
    apiVersion  = "csdr.alibabacloud.com/v1beta1"
    kind        = "BackupLocation"
    metadata = {
      name = "<yourBackuplocationName>"
      namespace = "csdr"
    }
  
    spec = {
      backupSyncPeriod = "0s"
      config = {
        network = "internal"
        region  = "cn-beijing"
      }
      objectStorage = {
        bucket 	= "<cnfs-oss-yourBucketName>"
        prefix 	= "<subDir>"
      }
      provider 	= "alibabacloud"  
    }
  }
  wait {
    fields = {
      "status.phase" = "Available"
    }
  }

  timeouts {
    create = "10m"
  }
}

Parameter	Required	Description
name	Yes	The name of the backup vault. The name must comply with the Kubernetes naming convention.
network	Yes	The network mode that is used to access the specified Object Storage Service (OSS) bucket. Valid values: internal: internal mode. All associated clusters must be deployed in the same region as the OSS bucket. public: public mode. This mode does not have region limits.
region	Yes	The region where the OSS bucket resides.
bucket	Yes	The name of the OSS bucket. Make sure that the OSS bucket name exists and starts with cnfs-oss-.
prefix	No	The subdirectory in the OSS bucket. If you specify this parameter, backups are stored in the specified subdirectory.

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

View output

  # kubernetes_manifest.backuplocation-demo will be created
  + resource "kubernetes_manifest" "backuplocation-demo" {
      + manifest = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "BackupLocation"
          + metadata   = {
              + name      = "<yourBackuplocationName>"
              + namespace = "csdr"
            }
          + spec       = {
              + backupSyncPeriod = "0s"
              + config           = {
                  + network = "internal"
                  + region  = "cn-beijing"
                }
              + objectStorage    = {
                  + bucket = "<cnfs-oss-yourBucketName>"
                  + prefix = "<subDir>"
                }
              + provider         = "alibabacloud"
            }
        }
      + object   = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "BackupLocation"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "<yourBackuplocationName>"
              + namespace                  = "csdr"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + spec       = {
              + accessMode          = (known after apply)
              + backupSyncPeriod    = "0s"
              + config              = {
                  + network = "internal"
                  + region  = "cn-beijing"
                }
              + objectStorage       = {
                  + bucket = "<cnfs-oss-yourBucketName>"
                  + caCert = (known after apply)
                  + prefix = "<subDir>"
                }
              + provider            = "alibabacloud"
              + validationFrequency = (known after apply)
            }
        }

      + timeouts {
          + create = "10m"
        }

      + wait {
          + fields = {
              + "status.phase" = "Available"
            }
        }
    }

Run the following command to create a backup repository.

terraform apply

If the following output is returned, the backup repository has been created.

kubernetes_manifest.backuplocation-demo: Creating...
kubernetes_manifest.backuplocation-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Step 3: Create a backup job

You can select a backup type based on your scenario:

Application backup: Backs up applications that run in a cluster, including cluster resources and the persistent volumes used by the applications.
Data protection: Backs up only persistent volume data. The backed up resources include only persistent volume claims (PVCs) and persistent volumes (PVs).

For more information, see Scenarios for application backup and data protection.

Application backup

Create an instant backup job

Add the following content to the csdr.tf configuration file and modify the content as needed.

resource "kubernetes_manifest" "applicationbackup-demo" {
  manifest = {
    apiVersion  	= "csdr.alibabacloud.com/v1beta1"
    kind        	= "ApplicationBackup"
    metadata = {
      name		= "<yourApplicationBackupName>"
      namespace 	= "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
      }
    }
    spec = {
      includedNamespaces 	= ["default","default1"]
      includedResources 	= ["statefulset"]
      excludedResources 	= ["excludedResources"]
      labelSelector 		= {
        matchLabels 		= {
          "app" = "mysql-sts"
        }
      }
      pvBackup = {
        defaultPvBackup 	= "false"
      } 
      storageLocation 		= "<yourBackuplocationName>"
      ttl									= "720h0m0s"
      includeClusterResources = "false"
    }
  }
  
  # The backup is complete when the status.phase field changes to Completed.
  # The backup duration depends on the number of applications and the data volume of persistent volumes in the cluster.
  #wait {
  #  fields = {
  #    "status.phase" = "Completed"
  #  }
  #}

  #timeouts {
  #  create = "60m"
  #}
 }

Parameter	Required	Description
`csdr.alibabacloud.com/backuplocations`	Yes	Information about the backup repository where the backup is stored. The configuration must be the same as the backup repository's configuration.
`name`	Yes	The name of the instant backup job.
`includedNamespaces`	Yes	Specify a name for the namespace.
`includedResources`	No	The cluster resource types to include in the backup. Important To avoid unexpected results, configure only one of includedResources or excludedResources. If both are left empty, all resource types are backed up.
`excludedResources`	No	The cluster resource types to exclude from the backup. Important To avoid unexpected results, configure only one of includedResources or excludedResources. If both are left empty, all resource types are backed up.
`matchLabels`	No	The labels used to filter resources. Only resources with matching labels are backed up.
`includeClusterResources`	No	Specify whether to back up all cluster-level resources, such as StorageClasses, CustomResourceDefinitions (CRDs), and webhooks. `true`: back up all cluster-level resources. `false`: back up only cluster-level resources used by namespace-level resources in the specified namespaces. For example, when the system backs up a pod, the service account used by the pod is assigned a cluster role. In this case, the cluster role is automatically backed up. When the system backs up a CustomResource (CR), the corresponding CRD is backed up. Note By default, `IncludeClusterResources` is set to `false` for backup tasks created in the ACK console.
`defaultPvBackup`		Specifies whether to back up persistent volumes. Valid values: true: Backs up the application and its persistent volumes. false: Backs up only the application.
`storageLocation`	Yes	The name of the backup repository. Note If your cluster already uses Velero, join the DingTalk group (ID: 35532895) for assistance.
`ttl`	Yes	The retention period of the backup. Data cannot be restored after the backup expires. The format is `720h0m0s`. The value must be in the range of `24h0m0s` to `1572864h0m0s`.

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

View output

  # kubernetes_manifest.applicationbackup-demo will be created
  + resource "kubernetes_manifest" "applicationbackup-demo" {
      + manifest = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "ApplicationBackup"
          + metadata   = {
              + annotations = {
                  + "csdr.alibabacloud.com/backuplocations" = "{name:<yourBackuplocationName>,region:cn-beijing,bucket:<cnfs-oss-yourBucketName>,prefix:<subDir>,provider:alibabacloud}"
                }
              + name        = "<yourApplicationBackupName>"
              + namespace   = "csdr"
            }
          + spec       = {
              + excludedResources  = [
                  + "excludedResources",
                ]
              + includedNamespaces = [
                  + "default",
                  + "default1",
                ]
              + includedResources  = [
                  + "statefulset",
                ]
              + labelSelector      = {
                  + matchLabels = {
                      + app = "mysql-sts"
                    }
                }
              + pvBackup           = {
                  + defaultPvBackup = "false"
                }
              + storageLocation    = "vault-a"
              + ttl                = "720h0m0s"
            }
        }
      + object   = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "ApplicationBackup"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "<yourApplicationBackupName>"
              + namespace                  = "csdr"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + spec       = {
              + backupType              = (known after apply)
              + excludedNamespaces      = (known after apply)
              + excludedResources       = [
                  + "excludedResources",
                ]
              + includeClusterResources = (known after apply)
              + includedNamespaces      = [
                  + "default",
                  + "default1",
                ]
              + includedResources       = [
                  + "statefulset",
                ]
              + labelSelector           = {
                  + matchExpressions = (known after apply)
                  + matchLabels      = {
                      + app = "mysql-sts"
                    }
                }
              + orderedResources        = (known after apply)
              + pvBackup                = {
                  + defaultPvBackup  = false
                  + pvcList          = (known after apply)
                  + pvcSelector      = {
                      + matchExpressions = (known after apply)
                      + matchLabels      = (known after apply)
                    }
                  + snapshotPostRule = (known after apply)
                  + snapshotPreRule  = (known after apply)
                }
              + storageLocation     = "<yourBackuplocationName>"
              + ttl                 = "720h0m0s"
            }
        }
    }

Run the following command to create an instant backup job.

terraform apply

If the following output is returned, the instant backup job has been created.

kubernetes_manifest.applicationbackup-demo: Creating...
kubernetes_manifest.applicationbackup-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Create a backup schedule

Add the following content to the csdr.tf configuration file and modify the content as needed.

View configuration file

resource "kubernetes_manifest" "backupschedule-demo" {
  manifest = {
    apiVersion  	= "csdr.alibabacloud.com/v1beta1"
    kind        	= "BackupSchedule"
    metadata = {
      name		= "<yourBackupScheduleName>"
      namespace 	= "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
      }
    }
    spec = {
      schedule = "1 4 * * *"
      template = {
        includedNamespaces 			= ["default","default1"]
        includedResources 			= ["statefulset"]
        excludedResources 			= ["excludedResources"]
        labelSelector 				= {
          matchLabels 				= {
            "app" = "mysql-sts"
          }
        }
        pvBackup = {
          defaultPvBackup 			= "false"
        } 
        storageLocation 			= "<yourBackuplocationName>"
        ttl											= "720h0m0s"
        includeClusterResources = "false"
      }
    }
  }
  wait {
    fields = {
      "status.phase" = "Enabled"
    }
  }
  timeouts {
    create = "10m"
  }
}

Parameter	Required	Description
csdr.alibabacloud.com/backuplocations	Yes	Information about the backup repository where the backup is stored. The configuration must be the same as the backup repository's configuration.
name	Yes	The name of the backup schedule.
schedule	Yes	The schedule for the backup job to run. Use a cron expression. For information about the format, see How do I specify a backup cycle when I create a backup plan?.
includedNamespaces	Yes	Specify a name for the namespace.
includedResources	No	You can specify the cluster resource type. Important To avoid unexpected results, configure only one of includedResources or excludedResources. If both are left empty, all resource types are backed up.
excludedResources	No	The cluster resource types to exclude from the backup. Important To avoid unexpected results, configure only one of includedResources or excludedResources. If both are left empty, all resource types are backed up.
matchLabels	No	The labels used to filter resources. Only resources with matching labels are backed up.
includeClusterResources	No	Specify whether to back up all cluster-level resources, such as StorageClasses, CustomResourceDefinitions (CRDs), and webhooks. `true`: back up all cluster-level resources. `false`: back up only cluster-level resources used by namespace-level resources in the specified namespaces. For example, when the system backs up a pod, the service account used by the pod is assigned a cluster role. In this case, the cluster role is automatically backed up. When the system backs up a CustomResource (CR), the corresponding CRD is backed up. Note By default, `IncludeClusterResources` is set to `false` for backup tasks created in the ACK console.
defaultPvBackup	Yes	Specifies whether to back up persistent volumes. Valid values: true: Backs up the application and its persistent volumes. false: Backs up only the application.
storageLocation	Yes	The name of the backup repository. Note If your cluster already uses Velero, join the DingTalk group (ID: 35532895) for assistance.
ttl	Yes	The retention period of the backup. Data cannot be restored after the backup expires. The format is `720h0m0s`. The value must be in the range of `24h0m0s` to `1572864h0m0s`.

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

View output

  # kubernetes_manifest.backupschedule-demo will be created
  + resource "kubernetes_manifest" "backupschedule-demo" {
      + manifest = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "BackupSchedule"
          + metadata   = {
              + annotations = {
                  + "csdr.alibabacloud.com/backuplocations" = "{name:<yourBackuplocationName>,region:cn-beijing,bucket:<cnfs-oss-yourBucketName>,prefix:<subDir>,provider:alibabacloud}"
                }
              + name        = "<yourBackupscheduleName>"
              + namespace   = "csdr"
            }
          + spec       = {
              + schedule = "1 4 * * *"
              + template = {
                  + excludedResources  = [
                      + "excludedResources",
                    ]
                  + includedNamespaces = [
                      + "default",
                      + "default1",
                    ]
                  + includedResources  = [
                      + "statefulset",
                    ]
                  + labelSelector      = {
                      + matchLabels = {
                          + app = "mysql-sts"
                        }
                    }
                  + pvBackup           = {
                      + defaultPvBackup = "false"
                    }
                  + storageLocation    = "<yourBackuplocationName>"
                  + ttl                = "720h0m0s"
                }
            }
        }
      + object   = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "BackupSchedule"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "<yourBackupscheduleName>"
              + namespace                  = "csdr"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + spec       = {
              + backupAll = (known after apply)
              + paused    = (known after apply)
              + schedule  = "1 4 * * *"
              + template  = {
                  + backupType              = (known after apply)
                  + excludedNamespaces      = (known after apply)
                  + excludedResources       = [
                      + "excludedResources",
                    ]
                  + includeClusterResources = (known after apply)
                  + includedNamespaces      = [
                      + "default",
                      + "default1",
                    ]
                  + includedResources       = [
                      + "statefulset",
                    ]
                  + labelSelector           = {
                      + matchExpressions = (known after apply)
                      + matchLabels      = {
                          + app = "mysql-sts"
                        }
                    }
                  + orderedResources        = (known after apply)
                  + pvBackup                = {
                      + defaultPvBackup  = false
                      + pvcList          = (known after apply)
                      + pvcSelector      = {
                          + matchExpressions = (known after apply)
                          + matchLabels      = (known after apply)
                        }
                      + snapshotPostRule = (known after apply)
                      + snapshotPreRule  = (known after apply)
                    }
                  + storageLocation         = "<yourBackuplocationName>"
                  + ttl                     = "720h0m0s"
                }
            }
        }
      + timeouts {
          + create = "10m"
        }

      + wait {
          + fields = {
              + "status.phase" = "Enabled"
            }
        }
    }

Run the following command to create a backup schedule.

terraform apply

If the following output is returned, the backup schedule has been created.

kubernetes_manifest.backupschedule-demo: Creating...
kubernetes_manifest.backupschedule-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Related operations

Find backups created by a backup schedule.

Use labelSelector to find backups created by the backup schedule.

data "kubernetes_resources" "list-applicationbackup" {
  api_version 			= "csdr.alibabacloud.com/v1beta1"
  kind       				= "ApplicationBackup"
  namespace         = "csdr"
  label_selector    = "csdr/schedule-name=terraform-schedule-test"
}

# Return the output.
output "applicationbackup-name" {
  value = data.kubernetes_resources.list-applicationbackup.objects
}

Modify a backup schedule.

Set spec.paused to true to pause the backup schedule. Then, you can modify other fields in spec that define the backup behavior, such as spec.schedule. The following code provides an example.

resource "kubernetes_manifest" "backupschedule-demo" {
  manifest = {
    apiVersion          = "csdr.alibabacloud.com/v1beta1"
    kind                = "BackupSchedule"
    metadata = {
      name                              = "<yourBackcupscheduleName>"
      namespace         = "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
      }
    }
    spec = {
      # Pause the backup schedule.
      paused = "true" 
      # Modify the backup cycle of the backup schedule.
      schedule = "0 5 * * *"
      template = {
        includedNamespaces      = ["default","default1"]
        includedResources       = ["statefulset"]
        excludedResources       = ["excludedResources"]
        labelSelector                   = {
          matchLabels                   = {
            "app" = "mysql-sts"
          }
        }
        pvBackup = {
          defaultPvBackup       = "false"
        }
        storageLocation                 = "vault-a"
        ttl                                                                     = "720h0m0s"
        includeClusterResources = "false"
      }
    }
  }
  wait {
    fields = {
      # After the backup schedule is paused, the expected status is Paused.
      "status.phase" = "Paused"
    }
  }
  timeouts {
    create = "10m"
  }
}

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

kubernetes_manifest.backupschedule-demo: Refreshing state...

Terraform used the selected providers to generate the following execution plan. Resource actions
are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # kubernetes_manifest.backupschedule-demo will be updated in-place
  ~ resource "kubernetes_manifest" "backupschedule-demo" {
      ~ manifest = {
          ~ spec       = {
              ~ paused   = "false" -> "true"
              ~ schedule = "1 4 * * *" -> "0 5 * * *"
                # (1 unchanged attribute hidden)
            }
            # (3 unchanged attributes hidden)
        }
      ~ object   = {
          ~ spec       = {
              ~ paused    = false -> true
              ~ schedule  = "1 4 * * *" -> "0 5 * * *"
                # (2 unchanged attributes hidden)
            }
            # (3 unchanged attributes hidden)
        }

      ~ wait {
          ~ fields = {
              ~ "status.phase" = "Enabled" -> "Paused"
            }
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Run the following command to apply the changes to the backup schedule.

terraform apply

If the following output is returned, the backup schedule has been modified.

kubernetes_manifest.backupschedule-demo: Modifying...
kubernetes_manifest.backupschedule-demo: Modifications complete after 1s

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Data protection

Create an instant backup job

Add the following content to the csdr.tf configuration file and modify the content as needed.

resource "kubernetes_manifest" "applicationbackup-demo" {
  manifest = {
    apiVersion  	= "csdr.alibabacloud.com/v1beta1"
    kind        	= "ApplicationBackup"
    metadata = {
      name		= "<yourApplicationBackupName>"
      namespace 	= "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
    }
    spec = {
      backupType = "PvBackup"
      includedNamespaces 	= ["default","default1"]
      pvBackup = {
        pvcList           = [
        {
          namespace								= "default"
          name                  	= "pvc-nas"
        },
        {
          namespace								= "default1"
          name                		= "pvc-oss"
        }
        ]
        storageClassList 	= ["disk-essd","disk-ssd"]
      } 
      storageLocation 		= "<yourBackuplocationName>"
      ttl									= "720h0m0s"
    }
  }
  
  # The backup is complete when the status.phase field changes to Completed.
  # The backup duration depends on the number of applications and the data volume of persistent volumes in the cluster.
  #wait {
  #  fields = {
  #    "status.phase" = "Completed"
  #  }
  #}

  #timeouts {
  #  create = "60m"
  #}
 }
}

Run the following command to create a resource plan.
```
terraform plan
```

Run the following command to create an instant backup job.

terraform apply

If the following output is returned, the instant backup job has been created.

kubernetes_manifest.applicationbackup-demo: Creating...
kubernetes_manifest.applicationbackup-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Create a backup plan

Add the following content to the csdr.tf configuration file and modify the content as needed.

View configuration file

resource "kubernetes_manifest" "backupschedule-demo" {
  manifest = {
    apiVersion  	= "csdr.alibabacloud.com/v1beta1"
    kind        	= "BackupSchedule"
    metadata = {
      name		= "<yourBackupScheduleName>"
      namespace 	= "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
      }
    }
    spec = {
      schedule = "1 4 * * *"
      template = {
        includedNamespaces 			= ["default","default1"]
        pvBackup = {
          pvcList           = [
          {
            namespace								= "default"
            name                  	= "pvc-nas"
          },
          {
            namespace								= "default1"
            name                		= "pvc-oss"
          }
          ]
          storageClassList 	= ["disk-essd","disk-ssd"]
        } 
        storageLocation 			= "<yourBackuplocationName>"
        ttl											= "720h0m0s"
      }
    }
  }
  wait {
    fields = {
      "status.phase" = "Enabled"
    }
  }
  timeouts {
    create = "10m"
  }
}

Parameter	Required	Description
`csdr.alibabacloud.com/backuplocations`	Yes	Information about the backup repository where the backup is stored. The configuration must be the same as the backup repository's configuration.
`name`	Yes	The name of the instant backup job.
`schedule`	Yes	The schedule for the backup job to run. Use a cron expression. For information about the format, see How do I specify a backup cycle when I create a backup plan?.
`includedNamespaces`	Yes	Specify a name for the namespace.
`pvcList`	No	Specifies the PVCs of the PVs that you want to back up. `name` is the name of the PVC. Important If you specify both `pvcList` and `storageClassList`, the `storageClassList` configuration does not take effect. If you specify neither `pvcList` nor `storageClassList`, all PVs in the specified namespaces are backed up.
`storageClassList`	No	Specifies the types of PVs to back up. This is the name of the StorageClass.
`storageLocation`	Yes	The name of the backup repository. Note If your cluster already uses Velero, join the DingTalk group (ID: 35532895) for assistance.
`ttl`	Yes	The retention period of the backup. Data cannot be restored after the backup expires. The format is `720h0m0s`. The value must be in the range of `24h0m0s` to `1572864h0m0s`.

Run the following command to create a resource plan.
```
terraform plan
```

Run the following command to create a backup schedule.

terraform apply

If the following output is returned, the backup schedule has been created.

kubernetes_manifest.backupschedule-demo: Creating...
kubernetes_manifest.backupschedule-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Step 4: Create a restore job

Add the following content to the csdr.tf configuration file and modify the content as needed.

resource "kubernetes_manifest" "applicationrestore-demo" {
  manifest = {
    apiVersion  = "csdr.alibabacloud.com/v1beta1"
    kind        = "ApplicationRestore"
    metadata = {
      name        = "<yourApplicationRestoreName>"
      namespace   = "csdr"
      annotations = {
        "csdr.alibabacloud.com/backuplocations" = "{\"name\":\"<yourBackuplocationName>\",\"region\":\"cn-beijing\",\"bucket\":\"<cnfs-oss-yourBucketName>\",\"prefix\":\"<subDir>\",\"provider\":\"alibabacloud\"}"
      }
    }
    spec = {
      appRestoreOnly 		= "false"
      preserveNodePorts		= "true"
      includedNamespaces 	= ["default","default1"]
      includedResources 	= ["statefulset"]
      excludedResources 	= ["excludedResources"]
      convertedarg = [
      {
        convertToStorageClassType	= "alicloud-disk-topology-alltype"
        namespace									= "default"
        persistentVolumeClaim		= "pvc-nas"
      },
      {
        convertToStorageClassType	= "alicloud-disk-topology-alltype"
        namespace									= "default1"
        persistentVolumeClaim		= "pvc-oss"
      }
      ]
      backupName			= "<yourApplicationBackupName>"
      namespaceMapping			= {
        "<backupNamespace>" = "<restoreNamespace>"
      }
    }
  }
  
  # The restore is complete when the status.phase field changes to Completed.
  # The restore duration depends on the number of applications and the data volume of persistent volumes in the cluster.
  #wait {
  #  fields = {
  #    "status.phase" = "Completed"
  #  }
  #}

  #timeouts {
  #  create = "60m"
  #}
}

Parameter	Required	Description
csdr.alibabacloud.com/backuplocations	Yes	The backup vault that stores the backups. Make sure that the information is the same as the configuration of the backup vault.
name	Yes	The name of the restore task.
appRestoreOnly	No	This parameter is only effective for application backup tasks and does not take effect under data protection tasks. Specifies whether to restore only applications from backups that contain volume data. Persistent volume claims (PVCs), persistent volumes (PVs), and the relevant data are not restored. Valid values: true: restores only applications. If you want to change the data source where you back up applications, you need to manually create a PVC and PV, create a restore task, and then set this parameter to true. false: restores applications and the relevant volume data. Default value: false.
preserveNodePorts	No	This parameter is only effective for application backup tasks and does not take effect under data protection tasks. Specifies whether to retain the NodePort of the application. If the backup cluster and restore cluster use the same NodePort, set this parameter to false to change the NodePort to a random port. In other scenarios, set the parameter to true.
includedNamespaces	Yes	The name of the namespace to be restored. If you leave this parameter empty, all namespaces that have been backed up are restored.
includedResources	No	This parameter is only effective for application backup tasks and does not take effect under data protection tasks. The type of cluster resource to be included in the restore list. Important To ensure that backups are created as expected, specify only one of the following parameters: includedResources and excludedResources. If you leave both parameters empty, all resource types are backed up.
excludedResources	No	This parameter is only effective for application backup tasks and does not take effect under data protection tasks. The type of cluster resource to be excluded from the restore list. Important To ensure that backups are created as expected, specify only one of the following parameters: includedResources and excludedResources. If you leave both parameters empty, all resource types are backed up.
backupName	Yes	The name of the backup to be restored. If you use the scheduled backup feature, you need to specify the name of a backup that is created at a point in time. Example: <yourBackupScheduleName>-20221205225845.
namespaceMapping	No	This parameter maps the namespace in the backup cluster to a namespace in the restore cluster. Valid values: <backupNamespace>: the namespace of the backups in the backup cluster. <restoreNamespace>: the namespace to which resources are restored in the restore cluster. If you do not specify this parameter, the resources are restored to a namespace that uses the same name as the namespace in the backup cluster. Note If <restoreNamespace> does not exist, the system automatically creates one.
imageRegistryMapping	No	This parameter re-specifies the image repository address, and is applicable to all image configurations of the application that meet the criteria. Valid values: <oldImageRegistry>: The original image registry address of the application during backup. <newImageRegistry>: The target image registry address during restoration. If not set, the image configurations in the application will remain unchanged. Example: If an application in the original cluster uses the image `registry.cn-beijing.aliyuncs.com/my-registry/nginx:v1` and needs to be adjusted to `registry.cn-hangzhou.aliyuncs.com/my-registry/nginx:v1`, use the following configuration for this field: `namespaceMapping: "registry.cn-beijing.aliyuncs.com/my-registry/": "registry.cn-beijing.aliyuncs.com/my-registry/"`
convertedarg	No	The StorageClass conversion list. For volumes of the FileSystem type, such as OSS, NAS, CPFS, and local volumes, you can configure this parameter to convert the StorageClasses of their PVCs to the specified StorageClass during the restoration process. For example, you can convert NAS volumes to disk volumes. convertToStorageClassType: the desired StorageClass. Make sure that the StorageClass exists in the current cluster. You can specify only the disk or NAS StorageClass. namespace: the namespace of the PVC. persistentVolumeClaim: the name of the PVC. The above are the required parameters for the StorageClass conversion feature. During the StorageClass conversion, the following optional parameters are also supported for modifying the AccessModes configuration of the volumes. convertToAccessModes: The desired AccessModes list. Note If you want to restore volumes with AccessModes set to ReadWriteMany or ReadOnlyMany to disks, you must set AccessModes to ReadWriteOnce to avoid forced disk detachment. You can run the `kubectl -ncsdr describe <backup-name>` command to query the PVC information of a backup. In the returned `status.resourceList.dataResource.pvcBackupInfo` list, the dataType field displays the data type of the PVC, which can be FileSystem or Snapshot. The nameSpace and pvcName fields display the namespace and name of the PVC.

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

View output

  # kubernetes_manifest.applicationrestore-demo will be created
  + resource "kubernetes_manifest" "applicationrestore-demo" {
      + manifest = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "ApplicationRestore"
          + metadata   = {
              + annotations = {
                  + "csdr.alibabacloud.com/backuplocations" = "{name:<yourBackuplocationName>,region:cn-beijing,bucket:<cnfs-oss-yourBucketName>,prefix:subpath,provider:alibabacloud}"
                }
              + name        = "<yourApplicationRestoreName>"
              + namespace   = "csdr"
            }
          + spec       = {
              + appRestoreOnly     = "false"
              + backupName         = "<yourApplicationbackupName>"
              + convertedarg       = [
                  + {
                      + convertToStorageClassType = "alicloud-disk-topology-alltype"
                      + namespace                 = "default"
                      + persistentVolumeClaim     = "pvc-nas"
                    },
                  + {
                      + convertToStorageClassType = "alicloud-disk-topology-alltype"
                      + namespace                 = "default1"
                      + persistentVolumeClaim     = "pvc-oss"
                    },
                ]
              + excludedResources  = [
                  + "excludedResources",
                ]
              + includedNamespaces = [
                  + "default",
                  + "default1",
                ]
              + includedResources  = [
                  + "statefulset",
                ]
              + namespaceMapping   = {
                  + default = "default1"
                }
              + preserveNodePorts  = "true"
            }
        }
      + object   = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "ApplicationRestore"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "<yourApplicationRestoreName>"
              + namespace                  = "csdr"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + spec       = {
              + appRestoreOnly          = false
              + backupName              = "<yourApplicationbackupName>"
              + convertedarg            = [
                  + {
                      + convertToStorageClassType = "alicloud-disk-topology-alltype"
                      + namespace                 = "default"
                      + persistentVolumeClaim     = "pvc-nas"
                    },
                  + {
                      + convertToStorageClassType = "alicloud-disk-topology-alltype"
                      + namespace                 = "default1"
                      + persistentVolumeClaim     = "pvc-oss"
                    },
                ]
              + excludedNamespaces      = (known after apply)
              + excludedResources       = [
                  + "excludedResources",
                ]
              + existingResourcePolicy  = (known after apply)
              + includeClusterResources = (known after apply)
              + includedNamespaces      = [
                  + "default",
                  + "default1",
                ]
              + includedResources       = [
                  + "statefulset",
                ]
              + labelSelector           = {
                  + matchExpressions = (known after apply)
                  + matchLabels      = (known after apply)
                }
              + namespaceMapping        = {
                  + default = "default1"
                }
              + preserveNodePorts       = true
              + pvRestore               = {
                  + pvcPrefix = (known after apply)
                  + pvcSuffix = (known after apply)
                }
              + pvRestoreOnly           = (known after apply)
            }
        }
    }

Run the following command to create a restore job.

terraform apply

The following response indicates that the restore job was created successfully.

kubernetes_manifest.applicationbackup-demo: Creating...
kubernetes_manifest.applicationrestore-demo: Creation complete after 1s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Step 5: Delete resources from the backup center

Important

Other clusters may also use the backup vault that you created. Therefore, the backup center does not allow you to delete the BackupLocation object.

Delete a backup schedule

You can run the terraform destroy command to delete the BackupSchedule resource and stop scheduled backups.

Delete a backup or restore job

Add the following content to the csdr.tf configuration file and modify the content as needed.

resource "kubernetes_manifest" "deleterequest-demo" {
  manifest = {
    apiVersion  	= "csdr.alibabacloud.com/v1beta1"
    kind        	= "DeleteRequest"
    metadata = {
      name		= "<objectName-dbr>"
      namespace 	= "csdr"
    }
    spec = {
      deleteObjectName 	= "<objectName>"
      deleteObjectType	= "Backup"
    }
  }
}

Parameter	Required	Description
name	Yes	The name of the deletion request. To delete a backup task, specify the request name in the following format: ApplicationBackup name of the backup task+"-dbr". To delete a restore task, specify the request name in the following format: ApplicationRestore name of the restore task+"-dbr".
deleteObjectName	Yes	The name of the resource to be deleted.
deleteObjectType	Yes	The type of resource to be deleted. Valid values: "Backup": deletes the ApplicationBackup object and relevant resources of the backup task. "Restore": deletes the ApplicationRestore object and relevant resources of the restore task. Note Deleting backup tasks does not affect the backups that are created in the cluster. Deleting restore tasks does not affect the data that is restored.

Run the following command to create a resource plan.

terraform plan

If the following output is returned, the resource plan has been created.

View output

  # kubernetes_manifest.deleterequest-demo will be created
  + resource "kubernetes_manifest" "deleterequest-demo" {
      + manifest = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "DeleteRequest"
          + metadata   = {
              + name      = "<objectName-dbr>"
              + namespace = "csdr"
            }
          + spec       = {
              + deleteObjectName = "<objectName>"
              + deleteObjectType = "Backup"
            }
        }
      + object   = {
          + apiVersion = "csdr.alibabacloud.com/v1beta1"
          + kind       = "DeleteRequest"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "<objectName-dbr>"
              + namespace                  = "csdr"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + spec       = {
              + backupName       = (known after apply)
              + deleteObjectName = "<objectName>"
              + deleteObjectType = "Backup"
            }
        }
    }

Run the following command to delete the resource.
```
terraform apply
```
If the following output is returned, the deleterequest resource deletion request has been processed.
```
kubernetes_manifest.deleterequest-demo: Creating...
kubernetes_manifest.deleterequest-demo: Creation complete after 0s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```
Note
The system uses the deleterequest resource to delete the corresponding backup or restore job. After the job is deleted, the deleterequest resource is also automatically deleted.

References

To back up and restore applications using kubectl, see Back up and restore applications in a cluster using kubectl.
To back up and restore applications in the ACK console, see the following topics: