All Products
Search
Document Center

Object Storage Service:Usage notes

Last Updated:Jul 05, 2024

Before you enable OSS-HDFS, you must be familiar with the relationships between OSS-HDFS and the features of Object Storage Service (OSS). This way, you can properly use OSS-HDFS and prevent data loss.

Warning

After OSS-HDFS is enabled for a bucket, data that is written by using OSS-HDFS is stored in the .dlsdata/ directory of the bucket. To ensure the availability of OSS-HDFS or prevent data loss, do not perform write operations on the .dlsdata/ directory or objects in the directory by using methods that are not supported by OSS-HDFS. For example, do not perform the following write operations: rename the directory, delete the directory, and delete objects in the directory.

The following table describes the risks or considerations when you use OSS features that may perform write operations on the .dlsdata/ directory.

Feature

Risk

Description

References

Retention policy

Data deletion failures

We recommend that you do not enable OSS-HDFS and configure retention policies for a bucket at the same time.

For example, OSS-HDFS is enabled for a bucket for which a retention policy is configured. When you use methods supported by OSS-HDFS to delete an object from the .dlsdata/ directory, you receive a message that indicates the object is deleted. However, OSS actually retains the object if the deletion occurs within the retention period and cannot recognize and delete the object after the retention period ends.

Retention policies

Lifecycle rule

Data loss

To configure or modify a lifecycle rule to match all objects in a bucket for which OSS-HDFS is enabled, use the NOT element to exclude the objects that are stored in the .dlsdata/ directory. This prevents lifecycle rule-triggered object deletion operations or storage class conversion operations from affecting the read and write operations on data in OSS-HDFS.

lifecycle

If you want to manage the lifecycle of data stored in OSS-HDFS, use the automatic storage tiering feature of OSS-HDFS.

Versioning

Automatic data deletion failures and service exceptions

Do not enable OSS-HDFS and versioning for a bucket at the same time.

If OSS-HDFS and versioning are enabled for a bucket, OSS-HDFS may not work as expected. To ensure the stability of OSS-HDFS, you need to suspend versioning at the earliest opportunity and configure lifecycle rules to remove delete markers.

Delete a directory

Data loss

To maintain OSS-HDFS stability and prevent data loss, do not delete the .dlsdata/ directory or its subdirectories.

Delete directories

Delete objects

Data loss

To maintain OSS-HDFS stability and prevent data loss, do not delete objects from the .dlsdata/ directory.

Delete objects

Rename directories

Data loss

To maintain OSS-HDFS stability and prevent data loss, do not rename the .dlsdata/ directory of a bucket for which hierarchical namespace and OSS-HDFS are enabled.

Rename a directory

Rename objects

Data loss

To maintain OSS-HDFS stability and prevent data loss, do not rename objects in the .dlsdata/ directory.

Rename an object

Upload objects

Data loss

To maintain OSS-HDFS stability and prevent data loss, do not upload objects to the .dlsdata/ directory by using methods not supported by OSS-HDFS.

Simple upload

Change the storage classes of objects

Data access failures and billing rule changes

We recommend that you do not change the storage classes of objects in the .dlsdata/ directory of a bucket for which OSS-HDFS is enabled.

If you change the storage class of an object in the .dlsdata/ directory to Infrequent Access (IA), the object remains accessible by using OSS-HDFS. If you change the storage class of an object in the directory to Archive, Cold Archive, or Deep Cold Archive, the object cannot be directly accessed by using OSS-HDFS. You must restore the object before you can access it.

When you change the storage class of an object to IA, Archive, Cold Archive, or Deep Cold Archive, take note of consequential factors such as the minimum billable size (64 KB), minimum storage duration, and data retrieval fees.

Bucket policy

Data access failures, automatic data deletion failures, and unexpected billing

To maintain access to the .dlsdata/ directory and objects in the directory, do not select Deny Access for Authorized Operation when you configure a bucket policy for a bucket for which OSS-HDFS is enabled.

Bucket policies

RAM

Data access failures, automatic data deletion failures, and unexpected billing

When you enable OSS-HDFS for a bucket, the AliyunOSSDlsDefaultRole role is automatically created and the AliyunOSSDlsRolePolicy policy is attached to the role. To maintain access to the .dlsdata/ directory and objects in the directory, do not disable, modify, or delete the role and the attached policy.

Enable OSS-HDFS and grant access permissions

Bucket inventory

Data contamination

To maintain OSS-HDFS availability and prevent data contamination, do not set Inventory Path to .dlsdata/ when you create an inventory for a bucket for which OSS-HDFS is enabled.

Bucket inventory

Logging

Data contamination

To maintain OSS-HDFS availability and prevent data contamination, do not set Log Prefix to .dlsdata/ when you configure logging for a bucket for which OSS-HDFS is enabled.

Logging

ZIP package decompression

Data contamination and data loss

To maintain OSS-HDFS availability and prevent data contamination, do not set Destination Directory to .dlsdata/ when you configure a ZIP package extraction rule for a bucket for which OSS-HDFS is enabled.

ZIP package decompression

Important

OSS-HDFS stores HDFS data and auxiliary data in the .dlsdata/ directory of OSS buckets. The data consumes OSS storage capacity, and you are charged storage fees. For more information, see Storage capacity usage of OSS-HDFS.