Data Online Migration:Migrate data

Usage notes

• Data Online Migration accesses the source data address by using the public interfaces provided by the storage service provider of the source data address. The access behavior depends on the interface implementation of the storage service provider.

• When Data Online Migration is used for migration, it consumes resources at the source and the destination data addresses. This may interrupt your business. To ensure business continuity, we recommend that you enable throttling for your migration tasks or run the migration tasks during off-peak hours after careful assessment.

• Before a migration task starts, Data Online Migration checks the files at the source and the destination data addresses. If a file at the source data address and a file at the destination data address have the same name, and the File Overwrite Method parameter of the migration task is set to Yes, the file at the destination data address is overwritten during migration. If the two files contain different information and the file at the destination data address needs to be retained, we recommend that you change the name of one file or back up the file at the destination data address.

• If the source data address and destination data address are File Storage NAS endpoints in the same region, data migration is performed over an internal network of Alibaba Cloud even if you select public endpoints as the data addresses.

Limits

• If symbolic links exist at the source data address, the files or directories to which the symbolic links point are migrated to the destination data address. The migrated files are named after the symbolic links. Example:

  • If a symbolic link named a exists at Source Data Address A and the symbolic link points to the b.jpg file, the b.jpg file is migrated to the destination data address and renamed a.

  • If a symbolic link named b.txt exists at Source Data Address B and the symbolic link points to the c/ directory, the c/ directory is migrated to the destination data address and renamed b.txt/.

  For more information, see Configure symbolic links.

  Note

  If the files to be migrated contain symbolic links, the migration workload may be greater than estimated, and the migration may continue even if the displayed migration progress reaches 100%. The migration duration varies based on the actual migration workload.

• If the symbolic links at the source data address create a loop or a symbolic link of the directory type points to its upper-level directory, the source data continuously increases and cannot be completely migrated. For example, Symbolic Link a points to Symbolic Link b, Symbolic Link b points to Symbolic Link c, and Symbolic Link c points to Symbolic Link a. We recommend that you run the unlink command to remove the loop of these symbolic links.

• The following unconventional types of disk files cannot be migrated:

  • Block device file

  • Character device file

  • First in, first out (FIFO) file or pipeline file

  • Socket file

  • Other unrecognized types

• Empty directories that exist at the source data address are skipped during the migration.

• The system synchronizes the attributes of all files from the source data address to the destination data address during data migration. The attributes include permissions, the user group, and the last modification time. However, the system does not synchronize the attributes of the directories to which the files belong from the source data address to the destination data address. The directories include the root directory and the subdirectories of each level.

Step 1: Create a source data address

1. Log on to the Data Online Migration console as the Resource Access Management (RAM) user that you created.

   Important

   If you want to migrate data across accounts, we recommend that you log on to the console by using the account to which the source NAS file system belongs.

2. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.

3. In the Create Data Address panel, configure the parameters and click OK. The following table describes the parameters.

   Parameter	Required	Description
   Data Type	Yes	The type of data storage. Select NAS.
   Data Name	Yes	The name of the source data address. The name must be 3 to 63 characters in length. The name cannot contain special characters except for hyphens (-) and underscores (_).
   Data Region	Yes	The region in which the NAS file system resides. For a NAS file system on Alibaba Cloud, select the region in which the file system resides. For a NAS file system on a third-party platform, select the region of the virtual private cloud (VPC) to which the file system is mounted.
   NAS Type	Yes	The type of the source NAS file system. Valid values: Alibaba Cloud: a NAS file system that is created on Alibaba Cloud. Others: a NAS file system that is created on a third-party platform. You must mount the file system to an Alibaba Cloud VPC.
   File System	This parameter is required only if you select Alibaba Cloud as NAS Type.	The NAS file system whose data you want to migrate.
   Mount Point	This parameter is required only if you select Alibaba Cloud as NAS Type.	The mount target of the NAS file system. Important You can mount the NAS file system only to a target that resides in a VPC. Mount targets on the classic network are not supported.
   VPC	This parameter is required only if you select Others as NAT Type.	The VPC to which the NAS file system is mounted.
   Switches	This parameter is required only if you select Others as NAT Type.	The vSwitch to which the NAS file system is mounted.
   NAS Address	This parameter is required only if you select Others as NAT Type.	The private IP address of the third-party NAS server that resides in the VPC.
   Sub Folder	No	The subdirectory in which the data to be migrated is stored. If you do not specify a subdirectory, data in the root directory (/) is migrated by default. Important If you specify a subdirectory, make sure that the subdirectory exists on the NAS server. Otherwise, the data address fails to be created. Note If you select an Extreme NAS file system, the value of this parameter starts with share/ by default. If you select another type of file system, you can enter a value of your choice. If you select a file system that uses the Server Message Block (SMB) protocol, the value starts with myshare/ by default.
   Connection Method	This parameter is required only if you select Others as NAT Type.	The protocol type.
   Connection Password	This parameter is required only if you select Others as NAT Type.	Specify whether a password is required. Valid values: No Password: No password is required to access the NAS service. Use Password: A username and a password are required to access the NAS service. You must enter a valid username and a valid password.

   Note

   For information about the status of a data address, see the View the status of a data address section of this topic.

Step 2: Create a destination data address

• If the destination NAS data address is located in the same VPC as the source NAS data address, see Step 1: Create a source data address.

• If the destination NAS file system resides in a different VPC from the source NAS file system, you must configure the destination data address in the Data Online Migration console based on the following instructions:

  • Data Type: Select NAS.

  • Data Region: Select the region in which the source NAS file system resides.

  • NAS Type: Select Others.

  • VPC: Select the VPC to which the source NAS file system is mounted.

  • Switches: Select the vSwitch of the source NAS file system.

  • NAS Address: If you set the NAS Type parameter of the destination NAS data address to Alibaba Cloud, enter the mount target of the destination NAS file system. If you set the NAS Type parameter to Others, enter the private IP address of the destination NAS file system in the VPC.

  • For more information, see Step 1: Create a source data address.

Step 3: Create a migration job

1. In the left-side navigation pane, choose Data Online Migration > Migration Jobs. On the page that appears, click Create Job.

2. In the Create Job panel, read the terms of the migration service and select I agree to the above terms and activate the Data Transport service. Then, click Next.

3. In the Fee Reminder message, click Yes, Go Ahead.

4. In the Job Config step, configure the parameters and click Next. The following table describes the parameters.

   Parameter	Required	Description
   Job Name	Yes	Enter a task name. The name must meet the following requirements: It must be 3 to 63 characters in length. It can contain lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
   Source Data Address	Yes	Select the source data address that you created.
   Destination Data Address	Yes	Select the destination data address that you created.
   Specified Directory	No	Specify the directories whose files and subdirectories are included or excluded during migration. Valid values: Do not filter: migrates all data from the source data address. Exclude: does not migrate the files or subdirectories in the specified directories. For example, you want to migrate all the directories in the root_dir/ directory except root_dir/bad_sub1/ and root_dir/bad_sub2/. In this case, you can select this method and specify bad_sub1/ and bad_sub2/. Contain: migrates only the files and subdirectories in the specified directories. For example, you want to migrate only root_dir/good_sub1/ and root_dir/good_sub2/ in the root_dir/ directory. In this case, you can select this method and specify good_sub1/ and good_sub2/. Note A directory name can contain only digits and letters. If you use special characters, the migration job may fail. A directory name cannot start with a forward slash (/) or a backslash (\), and cannot contain double forward slashes (//), double periods (..), or double quotation marks ("). The character string that consists of all the specified directory names cannot exceed 10 KB in length. A directory name must end with a forward slash (/). For example, you can specify docs/ as the directory name. You can specify up to 20 directories of the Exclude or Contain type.
   Migration Type	Yes	Select the data migration mode. Valid values: Full: The system migrates all data from the source data address to the destination data address. You must configure the Start Time Point of File parameter. The system migrates all data whose last modification time is later than the specified time. After all data is migrated, the migration job ends. If you change the data at the source data address after the migration job is complete, you can submit another job to migrate all data. In this case, the system migrates only the changed data. Incremental: The system runs a migration job based on the values that you specified for the Migration Interval and Migration Times parameters. You must configure the Start Time Point of File parameter. During the first migration, the system migrates all data whose last modification time is later than the specified time. After the first migration is complete, incremental migration is performed based on the value that you specified for the Migration Interval parameter. For each incremental migration, the system migrates only the data that is created or modified after the previous migration starts and before the current migration starts. If you set the Migration Times parameter to N, full migration is performed once and then incremental migration is performed (N - 1) times. For example, the current time is 08:00, March 10, 2019 and you set the Migration Interval parameter to 1, the Migration Times parameter to 5, and the Start Time Point of File parameter to 08:00, March 5, 2019. When the first migration starts, the system migrates all files whose last modification time is between 08:00, March 5, 2019 and 08:00, March 10, 2019. In this example, the first migration requires one hour to complete and the second migration starts at 10:00, March 10, 2019. The duration from 08:00 to 10:00 consists of one hour for the first migration process and one hour for the migration interval. During the second migration, the system migrates the files whose last modification time is between 08:00, March 10, 2019 and 10:00, March 10, 2019. The migration job consists of one full migration and four incremental migrations. Important Before a full or incremental migration job starts, the system compares the files at the source data address with the files at the destination data address. If a source file has the same name as a destination file, the destination file is overwritten by the source file when one of the following conditions is met: The source file is updated later than the destination file. The size of the source file is different from the size of the destination file.
   Start Time Point of File	Yes	The time to filter data to be migrated. The system migrates the data that is created or modified after the specified time. Valid values: All: migrates all files. Assign: migrates only the files that are created or modified after the specified time. For example, if you set the time to 08:00:00, November 1, 2018, only the files that are created or modified after 08:00:00, November 1, 2018 are migrated.
   Migration Interval	This parameter is required only if you select Incremental as Migration Type.	Default value: 1. Unit: hours. The maximum value is 24.
   Migration Times	This parameter is required only if you select Incremental as Migration Type.	Default value: 1. The maximum value is 30.

5. In the Data Prediction section of the Performance step, configure the Data Size and File Count parameters.

   Note

   To ensure a successful migration job, we recommend that you estimate the volume of data that you want to migrate as accurately as possible. For more information, see the Estimate the amount of data that you want to migrate section of the "Preparations" topic.

6. Optional. In the Flow Control section of the Performance step, configure the Time Range and Max Flow(MB/s) parameters and click Add.

   Note

   To ensure business continuity, we recommend that you configure the Time Range and Max Flow(MB/s) parameters based on the workload peaks and troughs.

7. Click Create and wait until the migration job is complete.

Step 4: Verify data

View the status of a data address