What preparations are required before data migration? - Data Online Migration

This topic describes the required preparations for data migration.

Step 1: Create the destination bucket

Create a destination bucket to store the migrated data. For more information, see Create a bucket.

Step 2: Create and authorize a RAM user

Important

Use this RAM user to create roles and perform migration operations. We recommend creating this user in the Alibaba Cloud account that owns the source or destination bucket.
If you have not created a RAM user, see Quick start: Create a RAM user and grant permissions.

Log on to the RAM console. On the Users page, find your RAM user and click Add Permissions in the Actions column.

System policy: AliyunOSSImportFullAccess (Permissions for managing Online Migration Service).

Custom policy: This policy must include the ram:CreateRole, ram:CreatePolicy, ram:AttachPolicyToRole, and ram:ListRoles permissions.

For more information, see Create a custom policy. The following is an example policy:

{
    "Version":"1",
    "Statement":[
        {
            "Effect":"Allow",
            "Action":[
                "ram:CreateRole",
                "ram:CreatePolicy",
                "ram:AttachPolicyToRole",
                "ram:ListRoles"
            ],
            "Resource":"*"
        }
    ]
}

Step 3: Authorize the source bucket

Follow the steps that correspond to your source bucket's ownership.

Note

In this topic, "your account" refers to the account that you use to access the migration service console. A bucket "belongs to" your account if both are under the same Alibaba Cloud account.

Owned source bucket

Automatic authorization
We recommend using automatic authorization in the Data Online Migration console. For more information, see Step 2: Create a source data address of the Migrate data topic.
Manual authorization

Note

Use manual authorization in the following scenarios:

You want to use a single role to grant permissions on multiple source buckets.
The number of roles in your account is near the limit, and you need to avoid creating new ones.
One-click automatic authorization is unsuitable or unavailable.

1. Create a RAM role

As the RAM user you created, log on to the RAM console. On the Roles page, click Create Role.

Principal Type: Select Cloud Service.
Principal Name: Select Data Transport.
Role Name: Enter the RAM role name. The RAM role name must be in lowercase.

Select Alibaba Cloud Service as the trusted entity type

Select Data Transport and enter a role name

2. Authorize the source bucket

On the Roles page, find the RAM role that you created and click Grant Permission in the Actions column.

Custom policy: This policy must include the oss:List* and oss:Get* permissions.

For more information, see Create a custom policy. Use the following policy as a reference:

Note

Replace <mySrcBucket> with your source bucket's name.

For more information about OSS policies, see Common examples of RAM policies.

Important

If the source bucket contains objects encrypted with Key Management Service (KMS), also grant the AliyunKMSFullAccess system policy to the role.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*"
      ],
      "Resource": [
        "acs:oss:*:*:<mySrcBucket>",
        "acs:oss:*:*:<mySrcBucket>/*"
      ]
    }
  ]
}

External source bucket

1. Create a RAM role

As the RAM user you created, log on to the RAM console. On the Roles page, click Create Role.

Principal Type: Select Cloud Service.
Principal Name: Select Data Transport.
Role Name: Enter the RAM role name. The RAM role name must be in lowercase

Select Alibaba Cloud Service as the trusted entity type

Select Data Transport and enter a role name

2. Authorize the source bucket

Important

Adding a bucket policy overwrites any existing policy. To prevent this, ensure the new policy includes the contents of the previous one.

Log on to the OSS console by using the account that owns the source bucket.
In the left-side navigation pane, click Buckets and select the source bucket.
In the left-side navigation pane, choose Permission Control > Bucket Policy.
On the Add by Syntax tab, click Edit to add a custom bucket policy.

Grant the RAM role permissions to list and read all resources in the bucket.
Note
Use the following policy as a reference. Replace the following placeholders with your actual values: <otherSrcBucket> with the source bucket name , <otherUid> with the UID of the source bucket's owner account , <myUid> with the UID of the Alibaba Cloud account that you use to access the migration console , and <roleName> with the name of the role you created . For more information about OSS policies, see Common examples of RAM policies.
```
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Principal": [
         "arn:sts::<myUid>:assumed-role/<roleName>/*"
      ],
      "Resource": [
        "acs:oss:*:<otherUid>:<otherSrcBucket>",
        "acs:oss:*:<otherUid>:<otherSrcBucket>/*"
      ]
    }
  ]
}
```

3. KMS authorization

If the source bucket contains objects that are encrypted with KMS, also grant the AliyunKMSFullAccess system policy to the role.
If objects in the source bucket are encrypted with a custom KMS key, also grant the role access to that key:
1. Log on to the Key Management Service console and find the key.
2. Configure the Key Policy. Select Cross-account User and enter the ARN of the authorized entity. For more information, see Set a key policy.

Step 4: Authorize the destination bucket

Follow the steps that correspond to your destination bucket's ownership.

Owned destination bucket

Automatic authorization
We recommend using automatic authorization in the Data Online Migration console. For more information, see Step 3: Create a destination data address of the Migrate data topic.
Manual authorization
Authorize the destination bucket
On the Roles page, find the RAM role that you created and click Grant Permission in the Actions column.
- Custom Policy: This policy must include the oss:List*, oss:Get*, oss:Put*, and oss:AbortMultipartUpload permissions.
For more information, see Create a custom policy. Use the following policy as a reference:
Note
Replace <myDestBucket> with your destination bucket's name.
For more information about OSS policies, see Common examples of RAM policies.
Important
If the destination bucket uses server-side KMS encryption, also grant the AliyunKMSFullAccess system policy to the role.
```
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Resource": [
        "acs:oss:*:*:<myDestBucket>",
        "acs:oss:*:*:<myDestBucket>/*"
      ]
    }
  ]
}
```

External destination bucket

1. Authorize the destination bucket

Important

Adding a bucket policy overwrites any existing policy. To prevent this, ensure the new policy includes the contents of the previous one.

Log on to the OSS console by using the account that owns the destination bucket.
In the left-side navigation pane, click Buckets and select the destination bucket.
In the left-side navigation pane, choose Permission Control > Bucket Policy.
On the Add by Syntax tab, click Edit to add a custom bucket policy.
- Grant the RAM role permissions to list, read, and write all resources in the bucket.

Note

Use the following policy as a reference. Replace the following placeholders with your actual values: <otherDestBucket> with the destination bucket name , <otherUid> with the UID of the Alibaba Cloud account that owns the destination bucket , <myUid> with the UID of the Alibaba Cloud account that you use to access the migration console , and <roleName> with the name of the role you created . For more information about OSS policies, see Common examples of RAM policies.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Principal": [
         "arn:sts::<myUid>:assumed-role/<roleName>/*"
      ],
      "Resource": [
        "acs:oss:*:<otherUid>:<otherDestBucket>",
        "acs:oss:*:<otherUid>:<otherDestBucket>/*"
      ]
    }
  ]
}

2. KMS authorization

If the destination bucket is configured with server-side KMS encryption, also grant the AliyunKMSFullAccess system policy to the role.
If the server-side encryption for the destination bucket uses a custom KMS key, also grant the role access to that key:
1. Log on to the Key Management Service console and find the key.
2. Configure the Key Policy. Select Cross-account User and enter the ARN of the authorized entity. For more information, see Set a key policy.

Step 5: Restore archived source data

Before you create a migration task to migrate data of the Archive storage class, you must manually restore the data. Take note of the following items when you restore the data:

Before you create a source data address and a migration task, make sure that data of the Archive storage class is restored.
Specify the number of days during which the restored data remains in the restored state based on the amount of data that you want to migrate. This can prevent data from entering the archived state during data migration.
You may be charged for the restoration operation. The fee may be relatively high. For more information about the billing methods, contact the service provider that offers the source bucket.

Note

Data Online Migration does not restore data in archived objects at the source data address during data migration. The objects that are not restored or are being restored cannot be migrated.

For instructions on how to restore objects in Alibaba Cloud OSS, see Restore Objects.

Step 1: Create the destination bucket

Step 2: Create and authorize a RAM user

Step 3: Authorize the source bucket

Owned source bucket

Automatic authorization

Manual authorization

1. Create a RAM role

2. Authorize the source bucket

External source bucket

1. Create a RAM role

2. Authorize the source bucket

3. KMS authorization

Step 4: Authorize the destination bucket

Owned destination bucket

Automatic authorization

Manual authorization

Authorize the destination bucket

External destination bucket

1. Authorize the destination bucket

2. KMS authorization

Step 5: Restore archived source data