All Products
Search
Document Center

Elastic Compute Service:Create application-consistent snapshots

Last Updated:Sep 18, 2024

If your business has strict requirements on data consistency, you can enable the application-consistent snapshot feature when you create a snapshot-consistent group. The feature ensures that the data within applications is consistent and included in snapshots. This topic describes how to create application-consistent snapshots for Linux and Windows instances in the Elastic Compute Service (ECS) console or by using Alibaba Cloud SDK for Go.

Background information

If you enable the application-consistent snapshot feature when you create a snapshot-consistent group, the system creates application- or file system-consistent snapshots based on the actual situation. The following table describes the two types of snapshots.

Snapshot type

Description

Scenario

Application-consistent snapshot

  • When the system is creating application-consistent snapshots, the system temporarily pauses write operations to applications to ensure that all existing write operation results are synchronized to disks. This ensures data consistency on disks and in applications such as databases.

  • Application-consistent snapshots are identified by the APPConsistent:True tag.

  • Application-consistent snapshots created by using one of the following methods based on the operating system:

    • In Linux operating systems, the feature is performed by using custom shell scripts based on your applications. When you use custom scripts, Alibaba Cloud does not guarantee the application consistency effects.

    • In Windows operating systems, Windows provides built-in features such as Volume Shadow Copy Service (VSS) to perform the feature.

Suitable for applications that strictly require data consistency, especially databases and critical business systems. You can use an application-consistent snapshot to restore the applications at the earliest opportunity for disaster recovery.

File system-consistent snapshot

  • If you do not configure scripts or the scripts contain invalid settings, file system-consistent snapshots instead of application-consistent snapshots are created. File system-consistent snapshots ensure the consistency of file system data at the point of time when the snapshots are created.

  • File system-consistent snapshots are identified by the FsConsistent:True tag.

  • File-consistent snapshots can be created by using one of the following methods based on the operating system:

    • In Linux operating systems, if no application scripts are available, file system-consistent snapshots are created.

    • In Windows operating systems, if the Contain Writers by Default option is disabled, file system-consistent snapshots are created.

Suitable for scenarios in which you want to ensure the overall consistency of file systems, such as file servers and document management systems. This can meet backup requirements to ensure the integrity of file hierarchies.

Limits

  • Application-consistent snapshots are supported only by Enterprise SSDs (ESSDs) for which the multi-attach feature is disabled.

  • You can create application-consistent snapshots for cloud disks that are attached only to the same ECS instance.

Prerequisites

  • An ECS instance is created and runs one of the following operating system versions:

    • Windows: Windows Server 2012 R2 or later.

    • Linux: CentOS 7.6 or later, Ubuntu 18.04 or later, or Alibaba Cloud Linux 2.

  • Cloud Assistant Agent is installed on the ECS instance. For more information, see Install Cloud Assistant Agent.

  • If you want to use Alibaba Cloud SDK for Go to create application-consistent snapshots, take note of the following items:

Procedure

Create application-consistent snapshots in the ECS console

Step 1: Configure a RAM role and attach it to the ECS instance

  1. Log on to the Resource Access Management (RAM) console.

  2. Create a RAM role, grant the RAM role permissions to create application-consistent snapshots, and then attach the RAM role to the ECS instance. For more information, see the Create an instance RAM role and attach the instance RAM role to an ECS instance section in the "Grant ECS access to resources of other Alibaba Cloud services by using instance RAM roles" topic. Take note of the following parameters:

    • RAM Role Name: Example: AppSnapshotRoleName.

    • Selected Trusted Entity: The system sets this parameter to Alibaba Cloud Service.

    • Custom policy: Example: AppSnapshotPolicy is used. The following sample code provides an example of the policy content:

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "ecs:DescribeSnapshot*",
                      "ecs:CreateSnapshot*",
                      "ecs:TagResources",
                      "ecs:DescribeDisks"
                  ],
                  "Resource": [
                      "*"
                  ],
                  "Condition": {}
              }
          ]
      }

      This policy grants the permissions to query snapshot information, create snapshots, specify tags, and query disk information.

Step 2: Create application-consistent snapshots for the instance

Linux instance

On a Linux instance, compile custom shell scripts, including prescript.sh and postscript.sh, based on the application deployed on the instance to ensure application consistency. The following section describes how to create application-consistent snapshots for a Linux instance.

  1. Prepare the scripts based on the application on the ECS instance and upload the scripts to the ECS instance in advance.

    Script

    Description

    Path

    Type

    Permissions

    Content

    prescript.sh

    Used to suspend all write operations to an application before you perform operations that may affect the application. This prevents data inconsistency due to new data generated during the operations.

    /tmp/prescript.sh

    Shell

    Only the root user has read, write, and execute permissions on the scripts.

    The script content is compiled based on your business requirements. For more information, see the Create the prescript.sh and postscript.sh script files section of the "Best practices for creating application-consistent snapshots for a Linux instance on which MySQL is deployed" topic.

    postscript.sh

    After an operation that may affect an application, such as data backup, is complete, the postscript.sh script is run to restore the application, which allows write operations to resume.

    /tmp/postscript.sh

    For information about how to upload files to an ECS instance, see Use Workbench to upload a file to and download a file from an instance.

    Note

    If the script settings, such as the permissions, storage path, or script name, are invalid, file system-consistent snapshots instead of application-consistent snapshots are created.

  2. Go to the Instance page in the ECS console.

    1. Log on to the ECS console.

    2. In the left-side navigation pane, choose Instances & Images > Instances.

    3. In the top navigation bar, select the region where the ECS instance resides.image.png

  3. Find the instance from which you want to create snapshots and choose 更多 > Disk and Image > Create Snapshot-consistent Group in the Actions column.

  4. In the Create Snapshot-consistent Group dialog box, configure the parameters for the snapshot-consistent group.

    1. Select the cloud disks for which you want to create the snapshot-consistent group and configure other snapshot parameters.

    2. Configure the parameters in the Application-consistent Snapshot section.

      • If you select Enable Application-consistent Snapshot and Enable File System I/O Suspension and Resume and correctly configure scripts, application-consistent snapshots are created.

      • If you select Enable Application-consistent Snapshot and Enable File System I/O Suspension and Resume but do not configure scripts or incorrectly configure scripts, file system-consistent snapshots are created.

      Note

      If you do not install Cloud Assistant on the instance and you select Enable Application-consistent Snapshot, the Cloud Assistant plug-in is automatically installed on the instance.

  5. Click OK.

    After you create the snapshot-consistent group, a message indicating the Cloud Assistant command ID and the task ID (InvokeId) is displayed, as shown in the following figure. You can check whether application-consistent snapshots are created based on the task ID.命令执行id

Windows instance

Windows instances use VSS to ensure application consistency. This section describes how to enable the application-consistent snapshot feature for a Windows instance.

  1. Go to the Instance page in the ECS console.

    1. Log on to the ECS console.

    2. In the left-side navigation pane, choose Instances & Images > Instances.

    3. In the top navigation bar, select the region where the ECS instance resides.image.png

  2. Find the instance for which you want to enable the application-consistent snapshot feature and choose 更多 > Disk and Image > Create Snapshot-consistent Group in the Actions column.

  3. In the Create Snapshot-consistent Group dialog box, configure the parameters for the snapshot-consistent group.

    1. Select the cloud disks for which you want to create the snapshot-consistent group and configure other snapshot parameters.

    2. Configure the parameters in the Application-consistent Snapshot section.

      • If you select Enable Application-consistent Snapshot and Contain Writers by Default, application-consistent snapshots are created.

      • If you select only Enable Application-consistent Snapshot, file system-consistent snapshots are created.

      Note

      If you do not install Cloud Assistant on the instance and you select Enable Application-consistent Snapshot, the Cloud Assistant plug-in is automatically installed on the instance.

  4. Click OK.

    After the snapshot-consistent group is created, a message indicating the Cloud Assistant command ID and the task ID appears, as shown in the following figure. You can check whether application-consistent snapshots are created based on the task ID.命令执行id

Step 3: Check whether application-consistent snapshots are created

After you create the snapshot-consistent group, go to the ECS Cloud Assistant page and check whether application-consistent snapshots are created. Then, go to the Snapshots page and view information about the snapshot-consistent group and application-consistent snapshots that belong to the group.

  1. On the ECS Cloud Assistant page, check whether application-consistent snapshots are created.

    1. Log on to the ECS console.

    2. In the left-side navigation pane, choose Maintenance & Monitoring > Cloud Assistant.

    3. Click the Command Execution Result tab.

    4. Find the task ID obtained in the previous step in the Task ID column and click the task ID to view the execution results.

      image.png

      The return value of ExitCode is 0 as shown in the preceding figure. This indicates that application-consistent snapshots are created as expected on Cloud Assistant. In this case, the ID of the snapshot-consistent group appears in the command output.

      Note

      If the return value of ExitCode is not 0, an error occurs. Troubleshoot the issue based on the value in the ExitCode column. For more information, see the Error codes section of this topic.

  2. On the Snapshots page, view information about the created snapshot-consistent group and snapshots that belong to the group.

    1. In the left-side navigation pane, choose Storage & Snapshots > Snapshots.

    2. Click the Snapshot-consistent Groups tab. Find the snapshot-consistent group that you created and click the ID of the snapshot-consistent group to view snapshot details.

    3. In the Snapshot Information section, check whether a snapshot is an application-consistent snapshot or a file system-consistent snapshot based on the tag.

      • If the APPConsistent:True tag appears, an application-consistent snapshot is created.

        image.png

      • If the FsConsistent:True tag appears, a file system-consistent snapshot is created.

        image.png

Create application-consistent snapshots by using Alibaba Cloud SDK for Go

Step 1: Configure a RAM role and attach it to the ECS instance

You can call the AttachInstanceRamRole operation to attach a RAM role to the ECS instance. In this example, a RAM role named AppSnapshotRoleName is used.

Sample code of Alibaba Cloud SDK for Go:

package main
import (
    "fmt"
    "os"
    "github.com/aliyun/alibaba-cloud-sdk-go/services/ecs"
)

func main() {
		// Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables are configured in the code runtime. 
    // If the project code is leaked, the AccessKey pair may be leaked and the security of all resources in your account is compromised. The following sample code provides an example on how to use environment variables to obtain an AccessKey pair and use the AccessKey pair to call API operations. We recommend that you use Security Token Service (STS) tokens, which provide higher security. 
    client, err := ecs.NewClientWithAccessKey(
        "cn-hangzhou",       
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"),     
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")) 

    // Call the AttachInstanceRamRole operation to attach the specified RAM role to the instance.
    request := ecs.CreateAttachInstanceRamRoleRequest()
    request.Scheme = "https"

    request.RamRoleName = "AppSnapshotRoleName"          // Specify the name of the RAM role.
    request.InstanceIds = "[\"i-bp17r83nppqf141v****\"]" // Specify the ID of the instance.

    response, err := client.AttachInstanceRamRole(request)
    if err != nil {
        fmt.Print(err.Error())
    }

    fmt.Println(response.String())
}

A response similar to the following one is returned. For more information about the parameters in the response, see AttachInstanceRamRole.调用结果

Step 2: Call the RunCommand operation to create application-consistent snapshots for the instance

Linux instance

Call the RunCommand operation to use Cloud Assistant to create application-consistent snapshots for one or more Linux instances.

Sample code of Alibaba Cloud SDK for Go:

package main

import (
    "fmt"
    "os"
    "github.com/aliyun/alibaba-cloud-sdk-go/services/ecs"
)

func main() {
		// Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables are configured in the code runtime. 
    // If the project code is leaked, the AccessKey pair may be leaked and the security of all resources in your account is compromised. The following sample code shows how to use environment variables to obtain an AccessKey pair and use the AccessKey pair to call API operations. We recommend that you use STS tokens, which provide higher security. 
    client, err := ecs.NewClientWithAccessKey(
        "cn-hangzhou",    // Specify the region in which the instance resides.
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"),     
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"))

    // Call the RunCommand operation to create application-consistent snapshots for the instance. 
    request := ecs.CreateRunCommandRequest()
    request.Scheme = "https"

    request.Type = "RunShellScript"
    // Specify the command content used to create application-consistent snapshots. For more information, see the following description of the CommandContent parameter. 
    request.CommandContent = "acs-plugin-manager --exec --plugin app-snapshot-plugin --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableFsFreeze=true,-TimeoutInSeconds=30,-PreScriptPath=\"/tmp/prescript.sh\",-PostScriptPath=\"/tmp/postscript.sh\",-ExcludeDiskId=\"\",-Name=\"LinuxApp1\""
    request.InstanceId = &[]string{"i-bp17r83nppqf141v****"} // Specify the ID of the instance.

    response, err := client.RunCommand(request)
    if err != nil {
        fmt.Print(err.Error())
    }
    fmt.Printf("response is %#v\n", response)
}

Description of the CommandContent parameter:

  • acs-plugin-manager --exec --plugin app-snapshot-plugin: runs the app-snapshot-plugin Cloud Assistant plug-in.

  • --params=: the parameters of the plug-in. The following table describes the parameters.

    Parameter

    Type

    Required

    Description

    ExcludeDiskId

    String

    No

    The disks to exclude from the snapshots.

    Name

    String

    Yes

    The name of the snapshot-consistent group.

    Description

    String

    No

    The description of the snapshot-consistent group.

    RamRoleName

    String

    Yes

    The RAM role attached to the instance. For information about the name of the RAM role, see the Step 1: Configure a RAM role and attach it to the ECS instance section of this topic.

    PreScriptPath

    String

    No

    The path in which the prescript.sh script is stored. Example: /tmp/prescript.sh. The prescript.sh script must meet the following requirements:

    • Only the root user is granted the read, write, and execute permissions (chmod 700) on the script as the owner.

    • Compile the custom script content based on your business requirements.

    Important

    This parameter is required when you create application-consistent snapshots for Linux instances. If the script contains invalid settings, such as permissions, paths, or file names, file system-consistent snapshots instead of application-consistent snapshots are created.

    PostScriptPath

    String

    No

    The path in which the postscript.sh script is stored. Example: /tmp/postscript.sh. The postscript.sh script must meet the following requirements:

    • Only the root user is granted the read, write, and execute permissions (chmod 700) on the script as the owner.

    • Compile the custom script based on your business requirements.

    Important

    This parameter is required when you create application-consistent snapshots for Linux instances. If the script contains invalid settings, such as permissions, paths, or file names, file system-consistent snapshots instead of application-consistent snapshots are created.

    EnableFsFreeze

    Boolean

    No

    Specifies whether to enable Linux FsFreeze to put file systems into the read-only state before you create snapshots.

    Default value: True.

    TimeoutInSeconds

    Integer

    No

    The timeout period for I/O operations.

    Default value: 30. Unit: seconds.

    ScriptTimeoutInSeconds

    Integer

    No

    The timeout period for script execution.

    Default value: 1800. Unit: seconds.

A response similar to the following one is returned. For information about the parameters in the response, see RunCommand.linux

Windows instance

Call the RunCommand operation to use Cloud Assistant to create application-consistent snapshots for one or more Windows instances.

Sample code of Alibaba Cloud SDK for Go:

package main

import (
    "fmt"
  	"os"
    "github.com/aliyun/alibaba-cloud-sdk-go/services/ecs"
)

func main() {
    // Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables are configured in the code runtime. 
    // If the project code is leaked, the AccessKey pair may be leaked and the security of all resources in your account is compromised. The following sample code provides an example on how to use environment variables to obtain an AccessKey pair and use the AccessKey pair to call API operations. We recommend that you use STS tokens, which provide higher security. 
    client, err := ecs.NewClientWithAccessKey(
        "cn-hangzhou",       // Specify the region in which the instance resides.
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"),     
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"))

    request := ecs.CreateRunCommandRequest()
    request.Scheme = "https"

    request.Type = "RunBatScript"
    // Specify the command content used to create application-consistent snapshots. For more information, see the following description of the CommandContent parameter. 
    request.CommandContent = "acs-plugin-manager --exec --plugin app-snapshot-plugin-win --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableWriters=true,-Description=\"AppSnapshot\",-ExcludeDiskId=\"\",-Name=\"APPSnapshot-1\""
    request.InstanceId = &[]string{"i-bp11vqwgh574****"} // Specify the ID of the instance.
    request.Timeout = "1800"

    response, err := client.RunCommand(request)
    if err != nil {
        fmt.Print(err.Error())
    }
    fmt.Printf("response is %#v\n", response)
}

Description of the CommandContent parameter:

  • acs-plugin-manager --exec --plugin app-snapshot-plugin-win: runs the app-snapshot-plugin-win Cloud Assistant plug-in.

  • --params=: the parameters of the plug-in. The following table describes the parameters.

    Parameter

    Type

    Required

    Description

    ExcludeDiskId

    String

    No

    The disks to exclude from the snapshots.

    Name

    String

    Yes

    The name of the snapshot-consistent group.

    Description

    String

    No

    The description of the snapshot-consistent group.

    RamRoleName

    String

    Yes

    The RAM role attached to the instance. For information about the name of the RAM role, see the Step 1: Configure a RAM role and attach it to the ECS instance section of this topic.

    EnableWriters

    Boolen

    No

    Specifies whether to create application-consistent snapshots. Valid values:

    • true: creates application-consistent snapshots.

    • false: creates file system-consistent snapshots.

    Default value: true.

A response similar to the following one is returned. For information about the parameters in the response, see RunCommand.win查询结果

Step 3: Call the DescribeInvocationResults operation to check whether snapshots are created

Call the DescribeInvocationResults operation to check whether the Cloud Assistant command is run as expected.

Sample code of Alibaba Cloud SDK for Go:

package main

import (
    "fmt"
    "os"
    "github.com/aliyun/alibaba-cloud-sdk-go/services/ecs"
)

func main() {
    // Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables are configured in the code runtime. 
    // If the project code is leaked, the AccessKey pair may be leaked and the security of all resources in your account is compromised. The following sample code provides an example on how to use environment variables to obtain an AccessKey pair and use the AccessKey pair to call API operations. We recommend that you use STS tokens, which provide higher security. 
    client, err := ecs.NewClientWithAccessKey(
        "cn-hangzhou",    // Specify the region ID of the instance.
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"),     
        os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")) 

    // Call the DescribeInvocationResults operation to check whether snapshots are created.
    request := ecs.CreateDescribeInvocationResultsRequest()
    request.Scheme = "https"

    request.InvokeId = "t-hz01qsegaxi****"        // The task ID of the command. You can obtain the task ID in the response in the previous step. 
    request.InstanceId = "i-bp17r83nppqf141v****" // The instance ID.
    request.CommandId = "c-hz01qsegaxd****"       // The command ID. You can obtain the command ID in the response in the previous step. 

    response, err := client.DescribeInvocationResults(request)
    if err != nil {
        fmt.Print(err.Error())
    }
    fmt.Printf("response is %#v\n", response)
}

A response similar to the following one is returned. For information about the parameters in the response, see DescribeInvocationResults.查看结果

  • ExitCode indicates the error code. A value of 0 indicates that the command was run as expected. A value other than 0 indicates that an error occurred. Troubleshoot the issue based on the error code. For more information, see the Error codes section of this topic.

  • Output indicates the command output, which is Base64-encoded.

    If the command was run successfully, the value of Output is Base64-encoded and contains the ID of the created snapshot-consistent group, as shown in the following example:

    [snapshotgroup="ssg-bp170v57ca9j01jb****"][message="Finish whole Processes of Snapshot successfully"]

Error codes

Error code (ExitCode)

Description

0

The application-consistent snapshots were created as expected.

1

One or more conditions are not met. One of the following errors may occur:

  • The disk category is not supported.

  • The snapshot name is invalid.

  • Network connectivity issues occurred.

  • No RAM role is attached to the instance.

  • The operating system of the instance is not supported.

2

The type or number of the parameters that follow the --params string is invalid.

3

One of the following errors may occur:

  • No ESSDs are attached to the instance.

  • The RAM role attached to the instance does not have permissions to call snapshot-related API operations.

4

The snapshot-consistent group cannot be created.

5

The snapshot-consistent group is not in the expected state.

6

The request to create the snapshot-consistent group timed out.

7

A disk snapshot in the snapshot-consistent group is not in the expected state.

8

Tags cannot be added to the snapshot.

9

The postscript.sh script failed to be executed.

10

The postscript.sh script failed to be executed.

11

The I/O of the file systems cannot be suspended.

12

The I/O of the file systems cannot be resumed.

13

No RAM role is attached to the instance.

14

The number of snapshots exceeded the upper limit.

15

The snapshot is not in the expected state.

16

The snapshot cannot be created because the previous snapshot is being created and the instant access feature is disabled.

255

An unknown error occurred.

References

You can create application-consistent snapshots for MySQL or SQL Server databases. For more information, see Best practices for creating application-consistent snapshots for a Linux instance on which MySQL is deployed or Best practices for creating application-consistent snapshots for a Windows instance on which SQL Server is deployed.