This topic provides answers to some commonly asked questions about cross-account and same-account replication, including cross-region replication (CRR) and same-region replication (SRR), in Object Storage Service (OSS).
Why am I unable to create a data replication rule?
Insufficient permissions
Same-account replication
For same-account replication, you need to grant the RAM role the permissions to perform replication operations on the source and destination buckets. For more information about how to create a RAM role and grant permissions to the RAM role, see Role types.
Cross-account replication
For cross-account replication, you must use Account A to grant the RAM role the permissions to replicate data in the source bucket and use Account B to grant the RAM role the permissions to receive replicated objects in the destination bucket. For more information about how to create a RAM role and grant permissions to the RAM role, see Procedure.
Versioning status inconsistency between source and destination buckets
The source bucket and destination bucket in a replication task must have the same versioning status: versioning-enabled or unversioned.
Invalid endpoint or AccessKey pair
When you create a data replication rule by using OSS SDKs or ossutil, check the following configurations:
Check whether the endpoints for the source and destination buckets are correct. For more information, see Regions and endpoints.
Check whether the AccessKey pair is correct. For more information, see View the information about AccessKey pairs of a RAM user.
Why are objects in the source bucket not replicated to the destination bucket?
If objects are not replicated to the destination bucket after you configure a data replication rule for the source bucket, troubleshoot the issue based on the following possible causes:
Replication completion time
Data is asynchronously replicated in near real time. The time required to replicate data from the source bucket to the destination bucket may range from several minutes to hours based on the size of the data. If the size of the objects to replicate is large, wait for a few minutes before you check the destination bucket for the replicas of the objects.
Source bucket configurations
Check whether the status of data replication is Enabled.
Check whether the prefix in the names of the objects that you want to replicate is correctly configured.
To replicate objects whose names contain a specific prefix from the source bucket to the destination bucket, specify the prefix in the data replication rule. For example, if only the log prefix is specified in the data replication rule, only objects whose names contain the log prefix, such as log/date1.txt and log/date2.txt, are replicated. Objects whose names do not contain the log prefix, such as date3.txt, are not replicated.
To replicate all objects from the source bucket to the destination bucket, do not specify a prefix.
Replication rule limit
If an object in a bucket is a replica that is created based on a data replication rule, the object cannot be replicated to a destination bucket based on another data replication rule. For example, if you configure a data replication rule to replicate objects from Bucket A to Bucket B and another replication rule to replicate objects from Bucket B to Bucket C, objects that are replicated from Bucket A to Bucket B are not replicated from Bucket B to Bucket C.
KMS-encrypted objects
If Key Management Service (KMS)-based encryption is configured for objects in the source or destination bucket, you must set the Replicate Objects Encrypted based on KMS parameter to Yes and configure the parameters shown in the following figure to replicate objects that are encrypted by using KMS:
CMK ID: The customer master key (CMK) that is used to encrypt objects in the destination bucket.
If you want to use a CMK to encrypt objects that are replicated to the destination bucket, you must create a CMK in the same region as the destination bucket in the KMS console. For more information, see Create a CMK.
RAM Role Name: The RAM role that is authorized to perform KMS-based encryption on the destination objects.
New RAM Role: A RAM role is created to encrypt the destination objects by using CMKs. The RAM role is in the
kms-replication-sourceBucketName-destinationBucketNameformat.AliyunOSSRole: The AliyunOSSRole role is used to perform KMS-based encryption on the destination objects. If the AliyunOSSRole role does not exist, OSS automatically creates the AliyunOSSRole role when you select this option.
NoteIf you create a RAM role or modify the permissions of an existing RAM role, make sure that you attach the
AliyunOSSFullAccesspolicy to the role. Otherwise, data may fail to be replicated.
You can call the HeadObject operation to query the encryption status of objects in the source bucket and the GetBucketEncryption operation to query the encryption status of the destination bucket.
How do I verify data consistency between the source and destination buckets?
You can run the following code to verify data consistency between the destination bucket and the source bucket after the replication task is complete:
import com.aliyun.oss.OSSClient;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.model.*;
import com.aliyun.oss.OSSException;
import com.aliyuncs.exceptions.ClientException;
public class Demo {
public static void main(String[] args) throws ClientException {
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
// Set srcEndpoint to the endpoint of the region in which the source bucket is located.
String srcEndpoint = "https://oss-cn-hangzhou.aliyuncs.com";
OSSClient srcClient = new OSSClient(srcEndpoint , credentialsProvider);
// Specify the name of the source bucket.
String srcBucketName = "src-replication-bucket";
// Set destEndpoint to the endpoint of the region in which the destination bucket is located.
String destEndpoint = "https://oss-cn-beijing.aliyuncs.com";
OSSClient destClient = new OSSClient(destEndpoint, credentialsProvider);
// Specify the name of the destination bucket.
String destBucketName = "dest-replication-bucket";
// If the source and destination buckets are unversioned, call the listObjectsV2 operation to list the objects that are replicated from the source bucket.
// If versioning is enabled or suspended for the source and destination buckets, call the listVersions operation to list the objects that are replicated from the source bucket.
ListObjectsV2Result result;
ListObjectsV2Request request = new ListObjectsV2Request(srcBucketName);
do {
result = srcClient.listObjectsV2(request);
for (OSSObjectSummary summary : result.getObjectSummaries())
{
String objectName = summary.getKey();
ObjectMetadata srcMeta;
try {
// Query the metadata of the objects that are replicated from the source bucket.
srcMeta = srcClient.headObject(srcBucketName, objectName);
} catch (OSSException ossException) {
if (ossException.getErrorCode().equals("NoSuchKey")) {
continue;
} else {
System.out.println("head src-object failed: " + objectName);
}
continue;
}
ObjectMetadata destMeta;
try {
// Query the metadata of the destination objects in the destination bucket.
destMeta = destClient.headObject(destBucketName, objectName);
} catch (OSSException ossException) {
if (ossException.getErrorCode().equals("NoSuchKey")) {
System.out.println("dest-object not exist: " + objectName);
} else {
System.out.println("head dest-object failed: " + objectName);
}
continue;
}
// Check whether the CRC-64 values of the source objects are the same as those of the destination objects.
Long srcCrc = srcMeta.getServerCRC();
String srcMd5 = srcMeta.getContentMD5();
if (srcCrc != null) {
if (destMeta.getServerCRC() != null) {
if (!destMeta.getServerCRC().equals(srcCrc)) {
System.out.println("crc not equal: " + objectName
+ " | srcCrc: " + srcCrc + " | destCrc: " + destMeta.getServerCRC());
}
continue;
}
}
// Check whether the MD5 values of the source objects are the same as those of the destination objects.
if (srcMd5!= null) {
if (destMeta.getContentMD5() != null) {
if (!destMeta.getContentMD5().equals(srcMd5)) {
System.out.println("md5 not equal: " + objectName
+ " | srcMd5: " + srcMd5 + " | destMd5: " + destMeta.getContentMD5());
}
continue;
}
}
// Check whether the ETag values of the source objects are the same as those of the destination objects.
if (srcMeta.getETag() == null || !srcMeta.getETag().equals(destMeta.getETag())) {
System.out.println("etag not equal: " + objectName
+ " | srcEtag: " + srcMeta.getETag() + " | destEtag: " + destMeta.getETag());
}
}
request.setContinuationToken(result.getNextContinuationToken());
request.setStartAfter(result.getStartAfter());
} while (result.isTruncated());
}
}
Does OSS support chained replication?
No, OSS does not support chained replication. For example, if a data replication rule is configured to replicate data from Bucket A to Bucket B and another data replication rule is configured to replicate data from Bucket B to Bucket C, data in Bucket A is replicated only to Bucket B and is not replicated to Bucket C.
If you want to replicate data from Bucket A to Bucket C, you must configure a data replication rule to replicate data from Bucket A to Bucket C.
An exception is that if historical data replication is enabled for Bucket A and Bucket B and historical data replication is in progress, data that is newly written to Bucket A may be detected by the historical data replication and replicated to Bucket C.
Does two-way synchronization between two buckets cause circular replication?
No, two-way synchronization does not cause circular replication. For example, if you configure two-way synchronization between Bucket A and Bucket B, data (historical and incremental data) that is replicated from Bucket A to Bucket B is not replicated again from Bucket B to Bucket A. Similarly, data (historical and incremental data) that is replicated from Bucket B to Bucket A is not replicated again from Bucket A to Bucket B.
Does a data replication rule synchronize lifecycle rule-based object deletions from the source bucket to the destination bucket?
It depends on how you specify Replication Policy of the replication rule.
If you set Replication Policy to Add/Change: When objects are deleted from the source bucket based on a lifecycle rule, OSS does not delete their copies from the destination bucket.
If you set Replication Policy to Add/Delete/Change: When objects are deleted from the source bucket based on a lifecycle rule, OSS deletes their copies from the destination bucket.
NoteIn the destination bucket, you may find objects with the same names as those that are deleted from the source bucket based on the lifecycle rule. This does not indicate that the Add/Delete/Change replication policy fails to take effect. You may manually write the objects with the same names as those that are deleted from the source bucket to the destination bucket.
Why is the replication progress of historical data displayed as 0% for a long period of time?
The replication progress of historical data is not updated in real time. The replication progress does not update until all objects are scanned. If a large number of objects are stored in the source bucket, such as hundreds of millions of objects, several hours are required before the replication progress of historical data is updated. If the replication progress of historical data is not updated, it does not mean that historical data is not replicated to the destination bucket.
You can check whether historical data in the source bucket is replicated to the destination bucket by viewing the storage capacity of the destination bucket and traffic usage, such as inbound and outbound traffic. For more information, see View the resource usage of a bucket.
Do versioning-suspended buckets support data replication?
No, versioning-suspended buckets do not support data replication. You can configure data replication rules only between two unversioned buckets or two versioning-enabled buckets.
If the destination bucket uses KMS to encrypt data, am I charged for calling API operations that are related to KMS encryption?
If the destination bucket uses KMS to encrypt data, you are charged for calling API operations that are related to KMS encryption. For more information about the fees, see Billing of KMS.
Can I disable a data replication rule?
Yes, you can disable a data replication rule by clicking Disable Replication next to the data replication rule.
After you disable the data replication rule, the replicated data is stored in the destination bucket. The incremental data in the source bucket is not replicated to the destination bucket.