Based on your business requirements, you may need to retain some infrequently accessed data in your Object Storage Service (OSS) bucket for a long period of time for compliance or archiving requirements, or delete data that is no longer needed in batches. You can configure lifecycle rules based on the last modified time to move cold data into an appropriate storage class or delete objects to reduce storage costs.
Scenarios
A medical institution stores medical records in OSS. These objects are occasionally accessed within six months after they are uploaded, and almost never after that. In this case, the data manager can configure a lifecycle rule to convert the storage class of the objects to Archive 180 days after they are uploaded.
A company stores the call records of its customer service hotline in OSS. These objects are frequently accessed within the first two months, occasionally after two months, and almost never after six months. Two years later, the objects no longer need to be stored. In this case, the data manager can configure a lifecycle rule to convert the storage class of these objects to Infrequent Access (IA) 60 days after they are uploaded and to Archive 180 days after they are uploaded, and then delete these objects 730 days after they are uploaded.
You want to delete all objects from your bucket. OSS allows you to manually delete up to 1,000 objects at a time. In this case, you can configure a lifecycle rule to delete all objects in the bucket the next day.
For more information about storage classes, see Overview.
Limits
Match conditions
Lifecycle rules support matching only based on prefixes and tags. Wildcard matching, suffix matching, and regular expression matching are not supported.
Part lifecycle
You cannot configure two or more lifecycle rules that contain a part lifecycle policy for objects whose names have overlapping prefixes. Examples:
Example 1
If you configure a lifecycle rule that contains a part policy for a bucket, you cannot configure another lifecycle rule that contains a part policy for any objects in the bucket.
Example 2
If you configure a lifecycle rule that contains a part policy for objects whose names contain the dir1 prefix in a bucket, you cannot configure another lifecycle rule that contains a part policy for objects whose names contain overlapping prefixes, such as dir1/dir2.
Storage class conversion
You cannot configure a lifecycle rule to convert the storage class of appendable objects to Cold Archive or Deep Cold Archive.
You cannot configure a lifecycle rule to convert the storage class of symbolic links to IA, Archive, Cold Archive, or Deep Cold Archive.
Usage notes
Number of lifecycle rules
A bucket can have up to 1,000 lifecycle rules. A lifecycle rule can contain both policies based on the last modified time and policies based on the last access time.
Overwrite semantics
The PutBucketLifecycle operation overwrites the existing configurations of a lifecycle rule of a bucket. For example, if a lifecycle rule named Rule1 is configured for a bucket and you want to configure another lifecycle rule named Rule2 for the bucket, perform the following operations:
Call the GetBucketLifecycle operation to query Rule1.
Add both Rule1 and Rule2 to the lifecycle rule configuration.
Call the PutBucketLifecycle operation to create Rule1 and Rule2 for the bucket.
Effective time
OSS loads a lifecycle rule within 24 hours after the rule is created. After the lifecycle rule is loaded, OSS runs the rule every day at 08:00:00 (UTC+8).
The interval between the last modified time of an object and the time when the lifecycle rule is run must be longer than 24 hours. For example, if you configure a lifecycle rule for a bucket to delete objects one day after they are uploaded, objects that are uploaded on July 20, 2020 are deleted on a different date based on the specific time when the objects are uploaded.
Objects uploaded before 08:00:00 (UTC+8) on July 20, 2020 are deleted from 08:00:00 (UTC+8) on July 21, 2020 to 08:00:00 (UTC+8) on July 22, 2020.
Objects uploaded after 08:00:00 (UTC+8) on July 20, 2020 are deleted from 08:00:00 (UTC+8) on July 22, 2020 to 08:00:00 (UTC+8) on July 23, 2020.
When you update a lifecycle rule, tasks that are scheduled to be performed based on the rule on the current day are suspended. We recommend that you do not frequently update lifecycle rules.
Completion time
Lifecycle rule that does not contain tags
For a lifecycle rule that is not based on tags, up to 1 billion lifecycle management actions, including object deletion, storage class conversion, and part expiration, can be completed within 24 hours in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), and Singapore. If the number of lifecycle management actions based on the lifecycle rule exceeds 1 billion, the time required to complete the actions may exceed 24 hours.
For a lifecycle rule that is not based on tags, up to 100 million lifecycle management actions can be completed within 24 hours in other regions. If the number of lifecycle management actions based on the lifecycle rule exceeds 100 million, the time required to complete the actions may exceed 24 hours.
Lifecycle rule that contains tags
For a lifecycle rule that is based on tags, up to 500 million lifecycle management actions, including object deletion, storage class conversion, and part expiration, can be completed within 24 hours in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), and Singapore. If the number of lifecycle management actions based on the lifecycle rule exceeds 500 million, the time required to complete the actions may exceed 24 hours.
For a lifecycle rule that is based on tags, up to 50 million lifecycle management actions can be completed within 24 hours in other regions. If the number of lifecycle management actions based on the lifecycle rule exceeds 50 million, the time required to complete the actions may exceed 24 hours.
If versioning is enabled for a bucket, a lifecycle management action on each object version in the bucket is counted towards the applicable limit.
Billing rules
If you use lifecycle rules to convert object storage classes or delete objects, you may be charged request fees and storage fees. For more information, see Fees related to lifecycle rules.
Lifecycle rules for a bucket for which OSS-HDFS is enabled
Lifecycle rules for OSS objects
To configure or modify a lifecycle rule based on the last modified time to match all objects in a bucket for which OSS-HDFS is enabled, use the NOT element to exclude the objects that are stored in the
.dlsdata/
directory. This prevents lifecycle rule-triggered object deletion or storage class conversion actions from applying to OSS-HDFS data and consequently affecting read and write operations on OSS-HDFS data.Lifecycle rules for Hadoop Distributed File System (HDFS) files
For example, you want to store frequently accessed objects in the Standard storage class and infrequently accessed objects in the IA, Archive, or Cold Archive storage class. In this case, you can configure a lifecycle rule to use the automatic storage tiering feature. For more information, see Automatic storage tiering of OSS-HDFS.
Lifecycle rule elements
Match conditions
Match by prefix: Objects and parts are matched by prefix. You can create multiple lifecycle rules to specify different prefixes. Each prefix must be unique. The naming rules for prefixes are the same as those for objects. For more information, see Object.
Match by tag: Objects are matched by tag key and tag value. You can specify multiple tags in a single lifecycle rule. The lifecycle rule applies to all objects that have the specified tags.OSS Lifecycle rules cannot match parts by tag.
NoteFor more information, see Add tags to an object.
Match by prefix and tag: Objects are matched by prefix and tag.
Match by bucket: All objects and parts that are stored in a bucket are matched.
NOT element: The NOT element specifies the name prefix and tag of objects for which you do not want a specific lifecycle rule to take effect. For more information about how to configure the NOT element, see NOT.
ImportantYou can configure only one NOT element for a lifecycle rule. You must specify only one prefix for the NOT element. In addition, you can configure up to one tag for the NOT element in a lifecycle rule.
Validity period or expiration date of objects and actions on expired objects
Validity period: Specify a validity period for objects in unversioned buckets and the current versions of objects in versioned buckets. Specify the action that you want OSS to perform on the objects after they expire. Objects that match a specific lifecycle rule are retained for the specified validity period after the objects are last modified. The specified action is performed on these objects after they expire.
Expiration date: Specify an expiration date for objects in unversioned buckets and the current versions of objects in versioned buckets. Specify the action that you want OSS to perform on these objects after they expire. All objects that are last modified before this date expire, and the specified action is performed on these objects after they expire.
Validity period of the previous versions of objects: Specify a validity period for previous versions of objects and the action that you want OSS to perform on the previous versions after they expire. Objects that match a specific lifecycle rule are retained for the specified validity period after the objects become previous versions. The specified action is performed on these objects after they expire.
You can configure lifecycle rules to convert the storage class of expired objects or delete expired objects. For more information, see Configuration elements.
Validity period or expiration date of parts and actions on expired parts
Validity period: Specify a validity period for parts. Parts that match a specific lifecycle rule are retained within the specified validity period and deleted after the parts expire.
Expiration date: Specify an expiration date for parts. Parts that are last modified before the expiration date are deleted.
Matching logic
Different prefixes
For example, the following objects are stored in a bucket:
logs/programl/log1.txt
logs/program2/log2.txt
logs/program3/log3.txt
doc/readme.txt
If the prefix specified in a lifecycle rule is logs/, the lifecycle rule takes effect for the first three objects whose names contain the logs/ prefix. If the prefix specified in a lifecycle rule is doc/readme.txt, the lifecycle rule takes effect only for the doc/readme.txt object.
A prefix can start with Chinese characters.
When the GetObject or HeadObject operation is performed on an object that matches a lifecycle rule, the x-oss-expiration
header is included in the response. The header contains two parameters: expiry-date
that indicates the expiration date of the object, and rule-id
that indicates the ID of the matched lifecycle rule.
Same prefixes and tags specified in multiple lifecycle rules
When objects that have the same name prefixes and tags match multiple lifecycle rules at the same time, lifecycle rules that are configured to delete objects take precedence over lifecycle rules that are configured to convert the storage class of objects. For example, both rule1 and rule2 described in the following table take effect for objects whose names contain the abc prefix and that have the a=1 tag. rule1 is configured to delete matched objects 20 days after they are last modified. rule2 is configured to convert the storage class of matched objects to Archive 20 days after they are last modified. If rule1 and rule2 are configured for a bucket at the same time, rule2 does not take effect.
rule | prefix | tag | action |
rule1 | abc | a=1 | Delete matched objects 20 days after they are last modified. |
rule2 | abc | a=1 | Convert the storage class of matched objects to Archive 20 days after they are last modified. |
Overlapping prefixes and the same tags specified in multiple lifecycle rules
For example, rule1 described in the following table converts the storage class of objects that have the a=1 tag to IA 10 days after they are last modified. rule2 described in the following table deletes objects whose names contain the abc prefix and that have the a=1 tag 120 days after they are last modified.
rule | prefix | tag | action |
rule1 | - | a=1 | Convert the storage class of matched objects to IA 10 days after they are last modified. |
rule2 | abc | a=1 | Delete matched objects 120 days after they are last modified. |
rule3 described in the following table converts the storage class of objects that have the a=1 tag to Archive 20 days after they are last modified. rule4 described in the following table converts the storage class of objects whose names contain the abc prefix and that have the a=1 tag to IA 30 days after they are last modified. If rule3 and rule4 are configured for a bucket at the same time, the storage class of objects whose names contain the abc prefix and that have the a=1 tag is converted to Archive 20 days after they are last modified based on rule3. Archive objects cannot be converted to IA objects. Therefore, rule4 does not take effect.
rule | prefix | tag | action |
rule3 | - | a=1 | Convert the storage class of matched objects to Archive 20 days after they are last modified. |
rule4 | abc | a=1 | Convert the storage class of matched objects to IA 30 days after they are last modified. |
NOT
If you configure multiple lifecycle rules for a bucket and one of the lifecycle rules contains the NOT element, the actions specified by the rule that contains the NOT element are performed only on objects that match the rule. Examples:
Example 1
A lifecycle rule is configured to delete objects whose names contain the dir/ prefix in the examplebucket bucket 100 days after the objects are last modified.
Another lifecycle rule that contains the NOT element is configured to delete all objects whose names do not contain the dir/ prefix in the examplebucket bucket 50 days after the objects are last modified.
The following table describes the actions on the objects after the preceding lifecycle rules are applied.
Object
Action
Objects whose names contain the dir/ prefix
Delete matched objects 100 days after they are last modified.
Objects whose names do not contain the dir/ prefix
Delete matched objects 50 days after they are last modified.
Example 2
A lifecycle rule that contains the NOT element is configured to delete all objects that do not have TagA (key1:value1) in the examplebucket bucket 30 days after the objects are last modified.
Another lifecycle rule is configured to delete objects that have TagB (key2:value2) in the examplebucket bucket 50 days after the objects are last modified.
The following table describes the actions on the objects after the preceding lifecycle rules are applied.
Object
Action
All objects that do not have TagA or TagB
Delete matched objects 30 days after they are last modified.
Objects that have only TagA
Do not delete matched objects.
Objects that have only TagB
Delete matched objects 30 days after they are last modified.
Objects that have TagA and TagB
Delete matched objects 50 days after they are last modified.
Procedure
Use the OSS console
Use OSS SDKs
Use ossutil
Use the OSS API
FAQ
What do I do if the Set bucket lifecycle error, InvalidArgument, Days in the Transition action for StorageClass Archive must be more than the Transition action for StorageClass IA
error message is reported?
Does a lifecycle rule for a bucket apply to existing objects in the bucket?
How do I modify a configuration item in one or more lifecycle rules for a bucket?
How do I delete one or more lifecycle rules that are configured for a bucket?
Does OSS generate logs when the storage classes of objects are converted or when expired objects are deleted based on lifecycle rules?
Can I create a lifecycle rule that removes object delete markers and deletes current object versions?
Can I create a lifecycle rule that is based on the last modified time of objects to convert the storage class of an object from IA to Standard?
References
By default, OSS uses the upload time of an object as the last modified time of the object. Specific operations update the last modified time of the object. However, storage class conversion based on a lifecycle rule does not update the last modified time. For a list of operations that affect the LastModified attribute of OSS objects, see What are the operations that affect the LastModified attribute of OSS objects?
If you convert the storage class of an IA, Archive, Cold Archive, or Deep Cold Archive object and delete the object before the minimum storage duration ends, you are charged for the entire minimum storage duration. For more information, see How am I charged for objects whose storage duration is less than the minimum storage duration?
A lifecycle rule allows you to convert the storage classes of all objects or objects whose names contain a specific prefix in a bucket or delete such objects. To delete objects whose names contain a specific extension, you can run the rm command in ossutil. For more information, see rm.
If you want OSS to identify hot and cold data based on data access patterns and automatically move cold data into more cost-effective storage classes, you can configure lifecycle rules based on the last access time. For more information, see Lifecycle rules based on the last access time.
For more information about how to query the storage classes of all objects in a bucket, see List objects.