Parameters for online migration tasks - Data Online Migration

This topic describes the parameters that you need to configure when you use Data Online Migration to migrate data. This helps you smoothly and efficiently perform a data migration.

Important

The information about third-party cloud services in this topic is for reference only. Alibaba Cloud does not make guarantees or other forms of commitments for the performance and reliability of third-party cloud services, or the potential impacts of operations performed by using these services.

Alibaba Cloud OSS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from or to an Alibaba Cloud Object Storage Service (OSS) bucket.

Domain Name (OSS Endpoint in the console of the previous version)
The endpoint of the region in which the data address resides. For more information about the valid formats of OSS endpoints, see Regions, endpoints, and open ports. If you migrate data from a third-party cloud service to OSS, data is uploaded to OSS over an internal network.
Bucket (OSS Bucket in the console of the previous version)
The name of the source or destination OSS bucket. Make sure that the bucket name is not prefixed or suffixed with invalid characters such as spaces, line feeds, and tab characters.

Prefix (OSS Prefix in the console of the previous version)
The prefix of the data address. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
- If you want to migrate data from OSS, you can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated.
- If you want to migrate data to OSS, you can migrate the source data to the specified directory.
Authorize Role (available only in the console of the new version)
To ensure data security, we recommend that you create a Resource Access Management (RAM) role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and attach policies to the role. For more information, see Preparations.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and attach policies to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
Access Key Id and Access Key Secret (available only in the console of the previous version)
The AccessKey pair that is used to access the OSS bucket. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. You can use the AccessKey pair of an Alibaba Cloud account or a RAM user. You cannot use a Security Token Service (STS) token. For more information, see AccessKey Pair Management. The Authorize Role parameter is not supported in the console of the previous version. To ensure data security, we recommend that you create a RAM user, grant the required permissions to the RAM user, and then create an AccessKey pair of the RAM user for a data migration job. After the migration job is complete, you can delete the AccessKey pair.
- If you want to migrate data from OSS, you must grant the RAM user the permissions to query and read data from all resources in the OSS bucket from which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the source bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:ListObjects",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "oss:GetObject",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```
- If you want to migrate data to OSS, you must grant the RAM user full permissions on the OSS bucket to which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the destination bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:*",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the Alibaba Cloud OSS console. For more information, see List buckets.

Amazon S3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an Amazon Simple Storage Service (Amazon S3) bucket.

Note

If access to the Amazon S3 bucket from which you want to migrate data is restricted, you cannot use Data Online Migration to migrate data from the bucket. Before you migrate data, disable the access restriction.

Domain Name (Endpoint in the console of the previous version)
The endpoint of the region in which the data address resides. For more information about the valid formats of Amazon S3 endpoints, see Amazon S3 endpoints in the Amazon S3 documentation. HTTPS and HTTP are supported.
Bucket (Bucket in the console of the previous version)
The name of the Amazon S3 bucket that stores the source data. Make sure that the bucket name is not prefixed or suffixed with invalid characters such as spaces, line feeds, and tab characters.
Prefix (Prefix in the console of the previous version)
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Access Key Id and Secret Access Key in the console of the previous version)
The access key pair that is used to access the Amazon S3 bucket. To ensure data security, we recommend that you create an Identity and Access Management (IAM) user, grant the required permissions to the IAM user, and then create an access key pair of the IAM user for a data migration task. After the migration task is complete, you can delete the access key pair. For more information, see Manage access keys for IAM users in the Amazon Web Services (AWS) IAM documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of objects that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of objects that you want to migrate.

Azure Blob

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Microsoft Azure Blob Storage (Azure Blob) container.

Domain Name
The uniform resource identifier (URI) of the resource in Azure Blob. Format: https://<Storage account>.blob.core.windows.net. For more information, see Resource URI Syntax in the Azure Blob documentation.
Storage Account, Secret, and Connection Strings
The name, access key, and connection strings of the storage account that has the read permissions on the source data. For more information, see View account access keys in the Azure Blob documentation.
Container
The name of the Azure Blob container that stores the source data. The name is case-sensitive and must be the same as what you specify when you create the container. For more information, see Create a container in the Azure Blob documentation.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.

Tencent COS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Tencent Cloud Object Storage (COS) bucket.

If you migrate data from a Tencent COS bucket to an OSS bucket, only Tencent COS V5 is supported. For more information, see Introduction in the Tencent COS documentation.

Parameters:

Domain Name (Region in the console of the previous version)
The abbreviation of the region in which the data address resides. For more information, see Regions and Access Endpoints in the Tencent COS documentation.
Bucket (Bucket in the console of the previous version)
The name of the COS bucket that stores the source data.
Prefix (Prefix in the console of the previous version)
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Secret Id and Secret Key in the console of the previous version)
The API key that is used to access the COS bucket. To ensure data security, we recommend that you create an API key for a data migration task and delete the API key after the migration task is complete. For more information, see Access Key in the Tencent Cloud Access Management (CAM) documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the Tencent COS console. For more information, see Viewing Data Overview in the Tencent COS documentation.

Qiniu Cloud KODO

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Qiniu Cloud Object Storage (KODO) bucket.

Domain Name (Endpoint in the console of the previous version)
The domain name of the KODO bucket that stores the source data. Format: http://Domain name. Examples:
```
http://oy4jki81y.bkt.clouddn.com
http://78rets.com1.z0.glb.clouddn.com
http://cartoon.u.qiniudn.com
```
Only the preceding three types of domain names are supported. For more information, see Domain name management in the Qiniu Cloud documentation.
Bucket (Bucket in the console of the previous version)
The name of the KODO bucket that stores the source data.
Prefix (Prefix in the console of the previous version)
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey (Access Key and Secret Key in the console of the previous version)
The AccessKey and the SecretKey that are used to access the KODO bucket. To ensure data security, we recommend that you create an AccessKey and a SecretKey for a data migration task and delete the AccessKey and the SecretKey after the migration task is complete. For more information, see QiniuToken in the Qiniu Cloud documentation.
Data Size and File Count (available only in the console of the previous version)
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the Qiniu Cloud KODO console. For more information, see Bucket overview in the Qiniu Cloud documentation.

Baidu Cloud BOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Baidu Object Storage (BOS) bucket.

Domain Name
The endpoint of the region in which the data address resides. For more information, see Regions and Endpoints in the BOS documentation.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The Access Key ID and the Secret Key that are used to access the BOS bucket. To ensure data security, we recommend that you create an Access Key ID and a Secret Key for a data migration task and delete the Access Key ID and the Secret Key after the migration task is complete. For more information, see How to Get AKSK in the BOS documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the BOS console. For more information, see Manage Bucket in the BOS documentation.

HTTP or HTTPS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an HTTP or HTTPS source.

Console of the new version

Inventory Path
The path of the list file. For more information, see the Step 1: Upload list files section of the "Preparations" topic.
Inventory Domain Name
The OSS endpoint or Amazon S3 endpoint of the region in which the data address resides.
- For more information about the valid formats of OSS endpoints, see Regions, endpoints, and open ports.
- For more information about the valid formats of Amazon S3 endpoints, see Amazon S3 endpoints in the Amazon S3 documentation.
Authorize Role (list file in OSS)
To ensure data security, we recommend that you create a RAM role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and attach policies to the role. For more information, see Preparations.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and attach policies to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
Inventory AccessKeyId and Inventory SecretAccessKey (list file in Amazon S3)
To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an access key pair of the IAM user for a data migration task. After the migration task is complete, you can delete the access key pair. For more information, see Manage access keys for IAM users in the Amazon S3 documentation.

Console of the previous version

File Path
The path of the list file. For more information, see the Step 2: Upload a list file section of the "Preparations" topic.

List Access Endpoint
The OSS endpoint of the region in which the data address resides. For more information about the valid formats of OSS endpoints, see Regions, endpoints, and open ports.
List Access AK and List Access SK
The AccessKey pair that is used to access the list file. To ensure data security, we recommend that you create a RAM user, grant the required permissions to the RAM user, and then create an AccessKey pair of the RAM user for a data migration job. After the migration job is complete, you can delete the AccessKey pair. For more information, see RAM policies.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of objects that you want to migrate.

USS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from UPYUN Storage Service (USS).

Domain Address
The endpoint of USS SDK. For more information, see Request method in the USS documentation.
Note
If you do not know the network type of your USS service, you can use http://v0.api.upyun.com.
Service Name
The name of the USS bucket that stores the source data.
Migration Folder
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Operator Name and Operator Secret
The operator name and the operator secret that are used to access the USS bucket. To ensure data security, we recommend that you create an operator and grant the operator the read permissions. For more information, see Storage management in the USS documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the USS console. For more information, see Object management in the USS documentation.

Kingsoft Cloud KS3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Kingsoft Standard Storage Service (KS3) bucket.

Domain Name
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the KS3 documentation.
Bucket
The name of the KS3 bucket that stores the source data.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The AccessKey that is used to access the KS3 bucket. To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an AccessKey of the IAM user for a data migration task. After the migration task is complete, you can delete the AccessKey. For more information, see Create an AccessKey for an IAM user in the Kingsoft Cloud documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the KS3 console. For more information, see Bucket in the KS3 documentation.

Huawei Cloud OBS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Huawei Cloud Object Storage Service (OBS) bucket.

Domain Name
The endpoint of the region in which the data address resides. For more information, see Regions and Endpoints in the Huawei Cloud documentation.
Bucket
The name of the OBS bucket that stores the source data.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The access key that is used to access the OBS bucket. To ensure data security, we recommend that you create an access key for a data migration task and delete the access key after the migration task is complete. For more information, see Managing Access Keys for an IAM User in the Huawei Cloud IAM documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the OBS console. For more information, see Viewing Bucket Information in the Huawei Cloud OBS documentation.

Volcano Engine TOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Volcano Engine Tinder Object Storage (TOS) bucket.

Domain Name
The S3 endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the Volcano Engine TOS documentation.
Bucket
The name of the TOS bucket that stores the source data.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The Access Key that is used to access the TOS bucket. To ensure data security, we recommend that you create an Access Key for a data migration task and delete the Access Key after the migration task is complete. For more information, see Manage Access Keys in the Volcano Engine TOS documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the TOS console. For more information, see View bucket information in the Volcano Engine TOS documentation.

UCloud US3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a UCloud US3 bucket.

Domain Name
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints in the UCloud US3 documentation.
Bucket
The name of the UCloud US3 bucket that stores the source data.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Public Key and Private Key
The API public key and the API private key that are used to access the UCloud US3 bucket. To ensure data security, we recommend that you create an API public key and an API private key for a data migration task and delete the API public key and the API private key after the migration task is complete. For more information, see Token management in the UCloud US3 documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You can obtain values for the two parameters in the UCloud US3 console. For more information, see View the information about a bucket in the UCloud US3 documentation.

Google Cloud Storage

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Google Cloud Storage bucket.

Bucket
The name of the Google Cloud Storage bucket that stores the source data.
Prefix
The prefix of the data address. You can specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Key File
The key file of the service account. For more information, see Create a service account key in the Google Cloud Storage documentation.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of objects that you want to migrate.