Object Storage Service:CRR across accounts

Cross-account replication involves buckets within two Alibaba Cloud accounts. Configuration of the trust policy and lowest level of permissions for the RAM role according to the following instructions is required.

1. Use Account A to perform the following operations:

   1. Create a normal service role.

      Important

      You can create a RAM role by using a RAM user. The RAM user must have the following permissions: ram:CreateRole, ram:GetRole, ram:ListPoliciesForRole, and ram:AttachPolicyToRole. However, it is risky to grant a RAM user permissions such as ram:CreateRole and ram:GetRole. You can use the Alibaba Cloud account with which the RAM user is associated to create a RAM role and grant the required permissions to the RAM role. Then, the RAM user can assume the RAM role that is created by the Alibaba Cloud account.

      When you create the role, set Select Trusted Entity to Alibaba Cloud Service, Role Type to Normal Service Role, and Select Trusted Service to OSS. For more information, see Create a regular service role.

      Note

      After the RAM role is created, record the Alibaba Cloud Resource Name (ARN) of the RAM role in the Basic Information section for subsequent permission authorization.

   2. Grant the RAM role the permissions to perform CRR on the source bucket.

      You can use one of the following methods to grant the permissions to the RAM role:

      Attach the required system policy to the RAM role

      Warning

      Attach the AliyunOSSFullAccess system policy to the RAM role. The RAM role to which the AliyunOSSFullAccess system policy is attached has the permissions to perform all operations on all buckets within the current Alibaba Cloud account. Proceed with caution when you attach the system policy to the RAM role.

      Attach a custom policy to the RAM role

      You can use a RAM policy to grant the RAM role the minimum permissions required to perform CRR on the source bucket.

      Note

      Replace src-bucket with the name of the source bucket based on your business requirements.

      {
         "Version":"1",
         "Statement":[
            {
               "Effect":"Allow",
               "Action":[
                  "oss:ReplicateList",
                  "oss:ReplicateGet"
               ],
               "Resource":[
                  "acs:oss:*:*:src-bucket",          	
                  "acs:oss:*:*:src-bucket/*"
               ]
            }
         ]
      }

      To replicate objects encrypted by Key Management Service (KMS) to the destination bucket, you need to attach the AliyunKMSFullAccess system policy to the RAM role. For more information about how to attach a system policy to a RAM role, see Grant permissions to a RAM role.

2. Use Account B to grant the RAM role the permissions to receive the objects that you want to replicate to the destination bucket.

   Method 1: (Recommended) Add in GUI

   1. Log on to the OSS console.

   2. In the left-side navigation pane, click Buckets. On the Buckets page, find the bucket named dest-bucket and click the bucket name.

   3. In the left-side navigation tree, choose Permission Control > Bucket Policy.

   4. On the Bucket Policy tab, click Add in GUI and then click Receive Objects to Replicate.

   5. In the Receive Objects to Replicate panel, set Obtain UID and RAM Role From to Source RAM role ARN for replication, and select the ARN of the RAM role that you created in Step 1 for Source RAM Role ARN for Replication.

   6. Click Generate Policy.

   Method 2: Add by Syntax

   1. In the left-side navigation tree, choose Permission Control > Bucket Policy.

   2. On the Bucket Policy tab, click Add by Syntax and then click Edit.

   3. In the code editor, enter the following bucket policy.

      Important

      • If you configure a bucket policy by specifying policy statements to grant the RAM role the required permissions, the new bucket policy overwrites the existing bucket policy. Make sure that the new bucket policy contains the content of the existing bucket policy. Otherwise, the authorization of the existing bucket policy may fail.

      • Replace the custom role name with the name of the role that you created based on your business requirements. If the created role name contains uppercase letters, you need to convert them to lowercase letters. For example, if the created role name is AliyunOssDrsRole, you need to convert AliyunOssDrsRole to aliyunossdrsrole. In the following example, the name of the destination bucket is dest-bucket, the UID of the account to which the source bucket belongs is 137918634953xxxx, and the UID of the account to which the destination bucket belongs is 111933544165xxxx.

      {
         "Version":"1",
         "Statement":[
            {
              "Effect":"Allow",
              "Action":[
                  "oss:ReplicateList",
                  "oss:ReplicateGet",
                  "oss:ReplicatePut",
                  "oss:ReplicateDelete"			
               ],
              "Principal": [
                  "arn:sts::137918634953xxxx:assumed-role/aliyunossdrsrole/*"		 
      		 ],
               "Resource":[
                  "acs:oss:*:111933544165xxxx:dest-bucket",          	
                  "acs:oss:*:111933544165xxxx:dest-bucket/*"
               ]
            }
         ]
      }

   4. Click Save. In the message that appears, click OK.

Create a CRR rule for the source bucket by using Account A.

1. Log on to the OSS console.

2. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the source bucket.

3. In the left-side navigation tree, choose Data Management > CRR.

4. On the CRR tab, click CRR.

5. In the CRR panel, configure the parameters. The following table describes the parameters.

   Section	Parameter	Description
   Configure Destination Bucket	Source Bucket	The region and name of the source bucket are displayed. You do not need to specify the parameter.
   	Destination Bucket	Select Specify a bucket that belongs to another Alibaba Cloud account, select a region from the Region drop-down list, and then enter the name of the destination bucket.
   Configure Replication Policy	Objects to Replicate	Select the objects that you want to replicate to the destination bucket. Valid values: All Objects in Source Bucket: OSS replicates all objects from the source bucket to the destination bucket. Objects with Specified Prefix: OSS replicates the objects whose names contain a specific prefix from the source bucket to the destination bucket. You can specify up to 10 prefixes.
   	Replication Policy	Configure the data replication mode. Valid values: Note After you create a data replication rule, changes to the x-oss-last-access-time attribute of the objects and the storage class conversion of objects in the source bucket because of lifecycle rules or the CopyObject operation are not synchronized to the destination bucket. Add/Change (applicable to disaster recovery): OSS replicates only object creation and update operations from the source bucket to the destination bucket. Important If this replication policy is applied, only objects that are uploaded or updated after the policy takes effect will be replicated to the destination bucket, and objects deleted from the source bucket will not be deleted from the destination bucket. This policy effectively prevents data loss in the destination bucket resulting from manual deletion or automated deletion triggered by lifecycle policies in the source bucket. Add/Delete/Change (applicable to scenarios in which multiple users or applications need to share and access the same dataset): OSS replicates object creation, update, and deletion operations from the source bucket to the destination bucket. Important Besides replication of newly uploaded and updated objects, this replication policy includes replication of deletion, which ensures data consistency. This policy is applicable to scenarios in which multiple users or applications need to share and access the same dataset. Any object deleted from the source bucket, either manually or through lifecycle policies, will also be deleted from the destination bucket. Objects cannot be recovered after they are deleted. If you perform multipart upload to upload an object to the source bucket, each uploaded part is replicated to the destination bucket. The complete object that is obtained by calling the CompleteMultipartUpload operation is also replicated to the destination bucket. For more information, see Use data replication with versioning.
   	Replicate Historical Data	Specify whether to replicate historical data (data that exists in the source bucket before you enable CRR) to the destination bucket. Yes: Historical data is replicated to the destination bucket. Important When historical data is replicated, objects that are replicated from the source bucket may overwrite objects that have the same names in the destination bucket. To prevent data loss, we recommend that you enable versioning for the source and destination buckets. No: OSS replicates only objects that are uploaded or updated after the CRR rule takes effect to the destination bucket.
   	Replicate Objects Encrypted Based on KMS	Specify whether to replicate objects encrypted based on KMS from the source bucket to the destination bucket. Valid values: Yes: If KMS-based encryption is configured for the destination bucket or objects in the source bucket, the objects are replicated to the destination bucket. If you set Replicate Objects Encrypted Based on KMS to Yes, you must specify the CMK ID parameter. Note You can call the HeadObject operation to query the encryption rules of objects in the source bucket and the GetBucketEncryption operation to query the encryption rules of the destination bucket. No: Objects that are encrypted based on KMS are not replicated to the destination bucket.
   	CMK ID	Specify the ARN of the KMS key. For more information about how to obtain the ARN of a KMS key, see Create a key in the KMS instance.
   	RAM Role Name	Select the RAM role that you created in Step 1 from the drop-down list.
   	Acceleration Type	Transfer Acceleration is supported only when the source and destination buckets are respectively located in a region in the Chinese mainland and a region outside the Chinese mainland. If you enable transfer acceleration, you are charged transfer acceleration fees. For more information, see Transfer acceleration fees.
   	RTC	Note RTC is available in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), and China (Shenzhen). RTC is available in the following regions: US (Silicon Valley) and US (Virginia). After Replication Time Control (RTC) is enabled, OSS replicates most of the objects that you uploaded to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. For more information, see RTC. You are charged when you enable RTC. For more information, see RTC traffic fees.

6. Click OK. In the message that appears, click Enable.

   • After you configure a CRR rule, you cannot modify or delete the rule.

   • After you configure a CRR rule, the replication task starts in 3 to 5 minutes. You can view the replication progress on the CRR tab of the source bucket.

   • In CRR, data is asynchronously replicated. The period of time that is required to replicate data from the source bucket to the destination bucket varies based on the amount of data. The period of time may range from a few minutes to a few hours.

CRR across accounts is supported only by using OSS SDK for Java, OSS SDK for Python, and OSS SDK for Go.

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.AddBucketReplicationRequest;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // We recommend that you do not save access credentials in the project code. Otherwise, access credentials may be leaked. As a result, the security of all resources in your account is compromised. In this example, access credentials are obtained from environment variables. Before you run the sample code, make sure that the environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the source bucket. Example: src-bucket. 
        String bucketName = "src-bucket";
        // Specify the name of the destination bucket. The destination bucket and the source bucket must belong to different Alibaba Cloud accounts. 
        String targetBucketName = "dest-bucket";
        // Specify the region in which the destination bucket is located. The source and destination buckets must be located in different regions. 
        String targetBucketLocation = "oss-cn-shanghai";

        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, credentialsProvider);

        try {
            AddBucketReplicationRequest request = new AddBucketReplicationRequest(bucketName);
            request.setTargetBucketName(targetBucketName);
            request.setTargetBucketLocation(targetBucketLocation);
            // Specify whether to replicate historical data. By default, historical data is replicated. In this example, this parameter is set to false, which indicates that historical data is not replicated. 
            request.setEnableHistoricalObjectReplication(false);
            // Specify the name of the RAM role that you want OSS to use to replicate data. The role must have the permissions to perform CRR on the source bucket and receive replicated objects in the destination bucket. 
            request.setSyncRole("yourRole");           
            //List prefixes = new ArrayList();
            //prefixes.add("image/");
            //prefixes.add("video");
            //prefixes.add("a");
            //prefixes.add("A");
            // Specify the prefixes that are contained in the names of the objects that you want to replicate. After you specify the prefixes, only objects whose names contain one of the prefixes are replicated to the destination bucket. 
            //request.setObjectPrefixList(prefixes);
            //List actions = new ArrayList();
            //actions.add(AddBucketReplicationRequest.ReplicationAction.PUT);
            // Specify that OSS replicates object creation and update operations from the source bucket to the destination bucket.
            //request.setReplicationActionList(actions);
            ossClient.addBucketReplication(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}        

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())
# Specify the endpoint of the region in which the source bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
# Specify the name of the source bucket. Example: src-bucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'src-bucket')
replica_config = ReplicationRule(
    # Specify the name of the destination bucket. The destination bucket and the source bucket must belong to different Alibaba Cloud accounts. 
    target_bucket_name='dest-bucket',
    # Specify the region in which the destination bucket is located. The destination bucket and the source bucket must be located in different regions. 
    target_bucket_location='oss-cn-shanghai',
    # Specify the name of the RAM role that you want OSS to use to replicate data. The role must have the permissions to perform CRR on the source bucket and receive replicated objects in the destination bucket. 
    sync_role_name='roleNameTest',
)

# Specify the prefixes that are contained in the names of the objects that you want to replicate.  
# prefix_list = ['prefix1', 'prefix2']
# Configure the data replication rule. 
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Specify that OSS replicates object creation and update operations from the source bucket to the destination bucket.
     # action_list=[ReplicationRule.P],
     # Specify the name of the destination bucket. The destination bucket and the source bucket must belong to different Alibaba Cloud accounts. 
     # target_bucket_name='dest-bucket',
     # Specify the region in which the destination bucket is located. The destination bucket and the source bucket must be located in different regions. 
     # target_bucket_location='yourTargetBucketLocation',
     # By default, historical data is replicated. In this example, this parameter is set to False, which indicates that historical data is not replicated. 
     # is_enable_historical_object_replication=False,
     # Specify the link that you want to use to transfer data during data replication. 
     # target_transfer_type='oss_acc',    
  #)

# Enable data replication. 
bucket.put_bucket_replication(replica_config)

package main

import (
    "encoding/xml"
    "fmt"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
    "os"
)

func HandleError(err error) {
    fmt.Println("Error:", err)
    os.Exit(-1)
}

// Enable data replication. 
func main() {
    // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
    provider, err := oss.NewEnvironmentVariableCredentialsProvider()
    if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
    }
    // Create an OSSClient instance. 
    // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
    client, err := oss.New("yourEndpoint", "", "", oss.SetCredentialsProvider(&provider))
    if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
    }
    // Specify the name of the source bucket. 
    srcbucketName := "yourSrcBucket"
    // Specify the name of the destination bucket. The destination bucket and the source bucket must belong to different Alibaba Cloud accounts. 
    destBucketName := "yourDestBucket"
    // Specify that objects whose names contain one of the following prefixes are replicated to the destination bucket: prefix_1 and prefix_2.  
    // To replicate all objects from the source bucket to the destination bucket, do not configure prefixes. 
    prefix1 := "prefix_1"
    prefix2 := "prefix_2"    
    prefixSet := oss.ReplicationRulePrefix{Prefix: []*string{&prefix1, &prefix2}}
    // Enable the RTC feature. 
    enabled := "enabled"
    reqReplication := oss.PutBucketReplication{
    Rule: []oss.ReplicationRule{
    {
    PrefixSet: &prefixSet,
    // Specify that OSS replicates object creation and update operations from the source bucket to the destination bucket.
    Action: "PUT",
    RTC:    &enabled,
    Destination: &oss.ReplicationRuleDestination{
    Bucket: destBucketName,
    // Specify the region in which the destination bucket is located. The source bucket and destination bucket must be located in different regions.     
    Location: "oss-cn-hangzhou",
    // Specify the link that you want to use to transfer data during data replication. In this example, this parameter is set to oss_acc, which indicates that the link used for transfer acceleration is used. 
    TransferType: "oss_acc",
    },
    // Specify whether to replicate historical data. By default, historical data is replicated. In this example, this parameter is set to disabled, which indicates that historical data is not replicated. 
    HistoricalObjectReplication: "disabled",
    // Specify the name of the RAM role that you want OSS to use to replicate data. The role must have the permissions to perform CRR on the source bucket and receive replicated objects in the destination bucket. 
    SyncRole:                    "yourRole",    
    },
    },
    }

    xmlBody, err := xml.Marshal(reqReplication)
    if err != nil {
    HandleError(err)
    }
    err = client.PutBucketReplication(srcbucketName, string(xmlBody))

    if err != nil {
    HandleError(err)
    }

    fmt.Println("Put Bucket Replication Success!")
}