Parameters for online migration jobs - Data Online Migration - Alibaba Cloud ドキュメントセンター

This topic describes the parameters that you need to configure when you use Data Online Migration to migrate data. This helps you smoothly and efficiently perform a data migration.

Important

The information about third-party products in this topic is for reference only. Alibaba Cloud does not make guarantees or other forms of commitments for the performance and reliability of the third-party tools, or the potential impacts of operations performed by using these tools.

Alibaba Cloud OSS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from or to an Object Storage Service (OSS) bucket.

Domain Name (OSS Endpoint in the console of the previous version)
The endpoint of the region in which the data address resides. For more information about the valid formats of OSS endpoints, see Regions and endpoints. If you migrate data from a third-party cloud service to OSS, data is uploaded to OSS over an internal network.
Bucket (OSS Bucket in the console of the previous version)
The name of the source or destination OSS bucket. Make sure that the bucket name is not prefixed or suffixed with invalid characters such as spaces, line feeds, and tab keys.

Prefix (OSS Prefix in the console of the previous version)
The prefix of the data address. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
- If you want to migrate data from OSS, you can specify a prefix to migrate only the files in the specified directory, which are the files with the specified prefix in their names. The file whose name is the prefix is not migrated.
- If you want to migrate data to OSS, you can specify a prefix to migrate the source data to the specified directory.
Authorize Role (available only in the console of the new version)
To ensure data security, we recommend that you create a Resource Access Management (RAM) role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and grant permissions to the role. For more information, see Migrate data within the same Alibaba Cloud account.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and grant permissions to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
Access Key Id and Access Key Secret (available only in the console of the previous version)
The AccessKey pair that is used to access the OSS bucket. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. You can use the AccessKey pair of an Alibaba Cloud account or a RAM user. You cannot use a Security Token Service (STS) token. For more information, see AccessKey Pair Management. The Authorize Role parameter is not supported in the console of the previous version. To ensure data security, we recommend that you create a RAM user, grant the required permissions to the RAM user, and then create an AccessKey pair of the RAM user for a data migration job. After the migration job is complete, you can delete the AccessKey pair.
- If you want to migrate data from OSS, you must grant the RAM user the permissions to list and read all resources in the OSS bucket from which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the source bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:ListObjects",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "oss:GetObject",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```
- If you want to migrate data to OSS, you must grant the RAM user full permissions on the OSS bucket to which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the destination bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:*",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the OSS console. For more information, see List buckets.

AWS S3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an AWS Simple Storage Service (AWS S3) bucket.

Note

If access to the AWS S3 bucket from which you want to migrate data is restricted, you cannot use Data Online Migration to migrate data from the bucket. Before you migrate data, disable the access restriction.

Domain Name (Endpoint in the console of the previous version)
The endpoint of the region in which the data address resides. For more information about the valid formats of AWS S3 endpoints, see Amazon S3 endpoints in the AWS S3 documentation. HTTPS and HTTP are supported.
Bucket
The name of the AWS S3 bucket that stores the source data. Make sure that bucket name does not contain invalid characters such as spaces, line feeds, and tab keys.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Access Key Id and Secret Access Key in the console of the previous version)
The access key that is used to access the AWS S3 bucket. To ensure data security, we recommend that you create an Identity and Access Management (IAM) user, grant the required permissions to the IAM user, and then create an access key of the IAM user for a data migration job. After the migration job is complete, you can delete the access key. For more information, see Managing access keys for IAM users in the AWS S3 documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of files that you want to migrate. You must set the Data Size and File Count parameters based on the actual amount of data and the actual number of files that you want to migrate.

Azure Blob

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Microsoft Azure Blob Storage (Azure Blob) container.

Storage Account, Secret, and Connection Strings
The name, access key, and connection string of the storage account that has the read permissions on the source data. For more information, see View account access keys in the Azure Blob documentation.
Container
The name of the Azure Blob container that stores the source data. The name is case-sensitive and must be the same as what you specified when you created the container. For more information, see Create a container in the Azure Blob documentation.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the Microsoft Azure console. For more information, see Container properties in the Azure Blob documentation.

Tencent Cloud COS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Tencent Cloud Object Storage (COS) bucket.

If you migrate data from a Tencent COS bucket to an OSS bucket, only Tencent COS V5 is supported. For more information, see Introduction in the Tencent COS documentation.

Region
The abbreviation of the name of the region in which the data address resides. For more information, see Regions and Access Endpoints in the Tencent COS documentation.
Bucket
The name of the COS bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Secret Id and Secret Key in the console of the previous version)
The API key that is used to access the COS bucket. To ensure data security, we recommend that you create an API key for a data migration job and delete the API key after the migration job is complete. For more information, see Access Key in the Tencent COS documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the Tencent COS console. For more information, see Viewing Data Overview in the Tencent COS documentation.

Qiniu Cloud KODO

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Qiniu Cloud Object Storage (KODO) bucket.

Domain Name (Endpoint in the console of the previous version)
The endpoint of the KODO bucket. The endpoints of buckets are in the http://<Domain name> format. The following examples show the valid types of endpoints.
```
http://oy4jki81y.bkt.clouddn.com
http://78rets.com1.z0.glb.clouddn.com
http://cartoon.u.qiniudn.com
```
Only the preceding three types of endpoints are supported. For more information, see Domain name management in the Qiniu Cloud documentation.
Bucket
The name of the KODO bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Access Key and Secret Key in the console of the previous version)
The Access Key and the Secret Key that are used to access the KODO bucket. To ensure data security, we recommend that you create an Access Key and a Secret Key for a data migration job and delete the Access Key and the Secret Key after the migration job is complete. For more information, see QiniuToken in the Qiniu Cloud documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the Qiniu Cloud console. For more information, see Bucket overview in the Qiniu Cloud documentation.

Baidu Cloud BOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Baidu Object Storage (BOS) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the BOS documentation.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access Key ID and Secret Access Key
The Access Key ID and the Secret Access Key that are used to access the BOS bucket. To ensure data security, we recommend that you create an Access Key ID and a Secret Access Key for a data migration job and delete the Access Key ID and the Secret Access Key after the migration job is complete. For more information, see How to Get AKSK. in the BOS documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the BOS console. For more information, see Bucket overview in the BOS documentation.

HTTP or HTTPS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an HTTP or HTTPS source.

Console of the new version

Inventory Path
The path of the list file. For more information, see the Step 1: Upload list files section of the "Preparations" topic.
Inventory Domain Name
The OSS endpoint or AWS S3 endpoint of the region in which the data address resides.
- For more information about the valid formats of OSS endpoints, see Regions and endpoints.
- For more information about the valid formats of AWS S3 endpoints, see Amazon S3 endpoints in the AWS S3 documentation.
Authorize Role (list file in OSS)
To ensure data security, we recommend that you create a RAM role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and grant permissions to the role. For more information, see Migrate data within the same Alibaba Cloud account.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and grant permissions to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
Inventory AccessKeyId and Inventory SecretAccessKey (list file in AWS S3)
To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an access key of the IAM user for a data migration job. After the migration job is complete, you can delete the access key. For more information, see Managing access keys for IAM users in the AWS S3 documentation.

Console of the previous version

File Path
The URL of the list file. For more information, see the Step 2: Upload list files section of the "Prepare for a migration job" topic.

List Access Endpoint
The OSS endpoint of the region in which the data address resides. For more information about the valid formats of OSS endpoints, see Regions and endpoints.
List Access AK and List Access SK
The AccessKey pair that is used to access the list file. To ensure data security, we recommend that you create a RAM user, grant the required permissions to the RAM user, and then create an AccessKey pair of the RAM user for a data migration job. After the migration job is complete, you can delete the AccessKey pair. For more information, see RAM policies.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You must set the Data Size and File Count parameters based on the actual amount of data and the actual number of files that you want to migrate.

USS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from UPYUN Storage Service (USS).

Domain Address
The USS endpoint that the OSS SDKs can access. For more information, see Request method in the USS documentation.
Note
You can use http://v0.api.upyun.com to query the network type of your USS service.
Service Name
The name of your USS bucket that stores the source data.
Migration Folder
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Operator Name and Operator Secret
The operator name and the operator secret that are used to access the USS bucket. To ensure data security, we recommend that you create an operator and grant the operator the read permissions. For more information, see Storage management in the USS documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the UPYUN Cloud console. For more information, see File management in the USS documentation.

Kingsoft Cloud KS3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Kingsoft Standard Storage Service (KS3) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the KS3 documentation.
Bucket
The name of the KS3 bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access Key ID and Secret Key
The Access Key ID and the Secret Access Key that are used to access the KS3 bucket. To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an Access Key ID and a Secret Access Key of the IAM user for a data migration job. After the migration job is complete, you can delete the Access Key ID and the Secret Access Key. For more information, see Create an AccessKey for an IAM user in the Kingsoft Cloud documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the KS3 console. For more information, see Basic operations in the KS3 documentation.

Huawei Cloud OBS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Huawei Object Storage Service (OBS) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the OBS documentation.
Bucket
The name of the OBS bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access Key ID and Secret Access Key
The Access Key ID and the Secret Access Key that are used to access the OBS bucket. To ensure data security, we recommend that you create an Access Key ID and a Secret Access Key for a data migration job. After the migration job is complete, you can delete the Access Key ID and the Secret Access Key. For more information, see Managing Access Keys for an IAM User in the Huawei Cloud documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the OBS console. For more information, see Viewing Bucket Information in the OBS documentation.

Volcano Engine TOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Volcano Engine Tinder Object Storage (TOS) bucket.

Domain Name
The S3 endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the TOS documentation.
Bucket
The name of the TOS bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The access key that is used to access the TOS bucket. To ensure data security, we recommend that you create an Access Key ID and a Secret Access Key for a data migration job. After the migration job is complete, you can delete the Access Key ID and the Secret Access Key. For more information, see Managing Access Keys for an IAM User in the Volcano Engine documentation.

UCloud US3

The section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a UCloud US3 bucket.

Region
The region in which the data address resides. For more information, see Regions and endpoints in the UCloud US3 documentation.
Bucket
The name of the UCloud US3 bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Public Key and Private Key
The API public key and the API private key that are used to access the UCloud US3 bucket. To ensure data security, we recommend that you create an API public key and an API private key for a migration job and delete the API public key and the API private key after the migration job is complete. For more information, see Token management in the UCloud US3 documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You can obtain values for the two parameters in the UCloud US3 console. For more information, see View the information about a bucket in the UCloud US3 documentation.

Google Cloud Storage

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Google Cloud Storage bucket.

Bucket
The name of the Google Cloud Storage bucket that stores the source data.
Prefix
The prefix of the data address, which indicates the directory in which the files that you want to migrate are located. Only the files in the specified directory can be migrated. The file whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Key File
The private key of a service account. For more information, see Create a private key for a service account in the Google Cloud Storage documentation.
Data Size and File Count
The amount of data and the number of files that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of files that you want to migrate.