use OSS SDK for Python to configure lifecycle rules to reduce storage costs - Object Storage Service

Not all data that is uploaded to Object Storage Service (OSS) buckets is frequently accessed. Rarely accessed data is stored in cold storage for compliance and archiving purposes. If you want to delete data stored in multiple OSS buckets at the same time, you can configure lifecycle rules based on the last modified time of the data. To reduce storage costs, we recommend that you store cold data and hot data separately. To achieve separate storage of hot data and cold data, you can configure OSS to automatically monitor data access modes, identify cold data, and then change the storage class of the cold data. In this scenario, you can also configure lifecycle rules based on the last access time of the data.

Usage notes

Before you configure lifecycle rules based on the last modified time or last access time of objects, make sure that you familiarize yourself with this feature. For more information, see Lifecycle rules based on the last modified time and Lifecycle rules based on the last access time.

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions, endpoints and open ports.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.
To configure lifecycle rules, you must have the oss:PutBucketLifecycle permission. To query lifecycle rules, you must have the oss:GetBucketLifecycle permission. To clear lifecycle rules, you must have the oss:DeleteBucketLifecycle permission. For more information, see Attach a custom policy to a RAM user.

Configure lifecycle rules

The following code provides examples of two lifecycle rules. One lifecycle rule is configured based on the last modified time of data. The other lifecycle rule is configured based on the last access time of data. After you configure lifecycle rules, you can modify the lifecycle rules based on your business requirements. For information about how to modify existing lifecycle rules, see Lifecycle rules based on the last modified time.

Configure a lifecycle rule based on the last modified time to change the storage classes of objects or delete objects

The following code provides an example on how to configure a lifecycle rule based on the last modified time to change the storage classes of objects or delete objects in the examplebucket bucket:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
import datetime
from oss2.models import (LifecycleExpiration, LifecycleRule, 
                        BucketLifecycle,AbortMultipartUpload, 
                        TaggingRule, Tagging, StorageTransition,
                        NoncurrentVersionStorageTransition,
                        NoncurrentVersionExpiration)

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Specify that objects expire three days after they are last modified. 
rule1 = LifecycleRule('rule1', 'tests/',
                      status=LifecycleRule.ENABLED,
                      expiration=LifecycleExpiration(days=3))

# Specify that objects created before the specified date expire. 
rule2 = LifecycleRule('rule2', 'tests2/',
                      status=LifecycleRule.ENABLED,
expiration = LifecycleExpiration(created_before_date=datetime.date(2023, 12, 12)))

# Specify that the parts expire three days after they are last modified. 
rule3 = LifecycleRule('rule3', 'tests3/',
                      status=LifecycleRule.ENABLED,
            abort_multipart_upload=AbortMultipartUpload(days=3))

# Specify that parts created before the specified date expire. 
rule4 = LifecycleRule('rule4', 'tests4/',
                      status=LifecycleRule.ENABLED,
                      abort_multipart_upload = AbortMultipartUpload(created_before_date=datetime.date(2022, 12, 12)))

# Specify that the storage classes of objects are changed to Infrequent Access (IA) 20 days after they are last modified, and to Archive 30 days after they are last modified. 
rule5 = LifecycleRule('rule5', 'tests5/',
                      status=LifecycleRule.ENABLED,
                      storage_transitions=[StorageTransition(days=20,storage_class=oss2.BUCKET_STORAGE_CLASS_IA),
                            StorageTransition(days=30,storage_class=oss2.BUCKET_STORAGE_CLASS_ARCHIVE)])

# Specify the tag that you want the lifecycle rule to match. 
tagging_rule = TaggingRule()
tagging_rule.add('key1', 'value1')
tagging_rule.add('key2', 'value2')
tagging = Tagging(tagging_rule)

# Specify that the storage classes of objects are changed to Archive 365 days after they are last modified.  
# Compared with the preceding rules, rule6 includes the tag condition to match objects. The rule takes effect for objects whose tagging configurations are key1=value1 and key2=value2. 
rule6 = LifecycleRule('rule6', 'tests6/',
                      status=LifecycleRule.ENABLED,
                      storage_transitions=[StorageTransition(created_before_date=datetime.date(2022, 12, 12),storage_class=oss2.BUCKET_STORAGE_CLASS_IA)],
                      tagging = tagging)

# rule7 is a lifecycle rule that applies to a versioning-enabled bucket. 
# Specify that the storage classes of objects are changed to Archive 365 days after they are last modified. 
# Specify that delete markers are automatically removed when they expire. 
# Specify that the storage classes of objects are changed to IA 12 days after they become previous versions. 
# Specify that the storage classes of objects are changed to Archive 20 days after they become previous versions. 
# Specify that objects are deleted 30 days after they become previous versions. 
rule7 = LifecycleRule('rule7', 'tests7/',
              status=LifecycleRule.ENABLED,
              storage_transitions=[StorageTransition(days=365, storage_class=oss2.BUCKET_STORAGE_CLASS_ARCHIVE)], 
              expiration=LifecycleExpiration(expired_detete_marker=True),
              noncurrent_version_sotrage_transitions = 
                    [NoncurrentVersionStorageTransition(12, oss2.BUCKET_STORAGE_CLASS_IA),
                     NoncurrentVersionStorageTransition(20, oss2.BUCKET_STORAGE_CLASS_ARCHIVE)],
              noncurrent_version_expiration = NoncurrentVersionExpiration(30))

lifecycle = BucketLifecycle([rule1, rule2, rule3, rule4, rule5, rule6, rule7])

bucket.put_bucket_lifecycle(lifecycle)

Configure a lifecycle rule based on the last modified time to change the storage classes of objects, excluding the objects whose names contain specific prefixes or that have specific tags

The following sample code provides an example on how to configure a lifecycle rule based on the last modified time. The lifecycle rule changes the storage classes of objects that meet the following conditions in the examplebucket bucket to IA 30 days after the objects are last modified: The names of the objects do not contain the log prefix and the objects do not have the tag whose key is tag1 and whose value is value1. The conditions are specified in the Not element of the filter node.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import LifecycleRule, BucketLifecycle, StorageTransition, LifecycleFilter, FilterNot, FilterNotTag

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

not_prefix = 'logs/not-prefix'
key = 'key1'
value = 'value1'
not_tag = FilterNotTag(key, value)
filter_not = FilterNot(not_prefix, not_tag)
filter = LifecycleFilter([filter_not])

# Create the lifecycle rule for storage class change. 
rule1 = LifecycleRule('mtime transition1', 'logs',
                      status=LifecycleRule.ENABLED,
                      storage_transitions=[StorageTransition(days=30, storage_class=oss2.BUCKET_STORAGE_CLASS_IA)],
                      filter=filter)

lifecycle = BucketLifecycle([rule1])

result = bucket.put_bucket_lifecycle(lifecycle)
print('The lifecycle configuration is successful. ' + str(result.status))

Configure a lifecycle rule based on the last modified time to change the storage classes of objects

The following code provides an example on how to configure a lifecycle rule based on the last access time to change the storage classes of objects in the examplebucket bucket:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import LifecycleRule, BucketLifecycle, StorageTransition, NoncurrentVersionStorageTransition

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Create a lifecycle rule and set ID to rule1. Specify that the storage classes of objects whose names contain the logs prefix and whose size is greater than 64 KB are changed to IA 30 days after the objects are last accessed. In addition, specify that the objects are still stored as IA objects when the objects are accessed again. 
rule1 = LifecycleRule('rule1', 'logs', status=LifecycleRule.ENABLED)
rule1.storage_transitions = [StorageTransition(days=30, storage_class=oss2.BUCKET_STORAGE_CLASS_IA, is_access_time=True, return_to_std_when_visit=False, allow_small_file=True)]

# Create a lifecycle rule and set ID to rule2. Specify that the storage classes of the previous versions of the objects whose names contain the dir prefix and whose size is greater than 64 KB are changed to IA 10 days after the objects are last accessed. In addition, specify that the storage classes of the objects are changed to Standard when the objects are accessed again. 
rule2 = LifecycleRule('rule2', 'dir', status=LifecycleRule.ENABLED)
rule2.noncurrent_version_sotrage_transitions = [NoncurrentVersionStorageTransition(10, oss2.BUCKET_STORAGE_CLASS_IA, is_access_time=True, return_to_std_when_visit=True, allow_small_file=False)]

lifecycle = BucketLifecycle([rule1, rule2])

# Create the lifecycle rules. 
result = bucket.put_bucket_lifecycle(lifecycle)

print('The lifecycle configuration is successful. ' + str(result.status))

Query lifecycle rules

The following code provides an example on how to query the lifecycle rules configured for the bucket named examplebucket:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Query the lifecycle rules. 
lifecycle = bucket.get_bucket_lifecycle()

for rule in lifecycle.rules:
    print('==========')
    # Query the IDs of the lifecycle rules. 
    print('id:', rule.id)
    # Query the prefix of the lifecycle rules. 
    print('prefix:', rule.prefix)
    # Query the status of the lifecycle rules. 
    print('status', rule.status)
    if rule.tagging is not None:
        # Query the tags of the lifecycle rules. 
        print('tagging:', rule.tagging)

    if rule.abort_multipart_upload is not None:
        if rule.abort_multipart_upload.days is not None:
            # Query the configurations of expired parts. 
            print('abort_multipart_upload days:', rule.abort_multipart_upload.days)
        else:
            print('abort_multipart_upload created_before_date:', rule.abort_multipart_upload.created_before_date)

    if rule.expiration is not None:
        if rule.expiration.days is not None:
            # Query the validity period of the lifecycle rules. 
            print('expiration days:', rule.expiration.days)
        elif rule.expiration.expired_detete_marker is not None:
            # Check whether expired delete markers are automatically removed. 
            print('expiration expired_detete_marker:', rule.expiration.expired_detete_marker)
        elif rule.expiration.created_before_date is not None:
            # Query the expiration date of the lifecycle rules. 
            print('expiration created_before_date:', rule.expiration.created_before_date)

    if len(rule.storage_transitions) > 0:
        storage_info = ''
        for storage_rule in rule.storage_transitions:
            if storage_rule.days is not None:
                storage_info += 'days={0}, storage_class={1} *** '.format(
                    storage_rule.days, storage_rule.storage_class)
            else:
                storage_info += 'created_before_date={0}, storage_class={1} *** '.format(
                    storage_rule.created_before_date, storage_rule.storage_class)
            # Check whether lifecycle rules are configured based on the last access time. Only OSS SDK for Python 2.16.1 and later support the feature. 
            storage_info += 'is_access_time={0}, return_to_std_when_visit={1} ***'.format(
                storage_rule.is_access_time, storage_rule.return_to_std_when_visit)
        print('storage_transitions:', storage_info)

    if len(rule.noncurrent_version_sotrage_transitions) > 0:
        noncurrent_storage_info = ''
        for storage_rule in rule.noncurrent_version_sotrage_transitions:
            noncurrent_storage_info += 'days={0}, storage_class={1} *** '.format(
                storage_rule.noncurrent_days, storage_rule.storage_class)
            # Check whether the storage classes of objects are changed to Standard when they are accessed again after their storage classes are changed to IA. Only OSS SDK for Python 2.16.1 and later support the feature. 
            noncurrent_storage_info += 'is_access_time={0}, return_to_std_when_visit={1} ***'.format(
                storage_rule.is_access_time, storage_rule.return_to_std_when_visit)
        # Query the rule that changes the storage classes of objects of previous versions. 
        print('noncurrent_version_sotrage_transitions:', noncurrent_storage_info)

    if rule.noncurrent_version_expiration is not None:
        # Query the expiration configurations for the previous versions of the objects. 
        print('noncurrent_version_expiration days:', rule.noncurrent_version_expiration.noncurrent_days)

Clear lifecycle rules

The following code provides an example on how to clear lifecycle rules configured for a bucket named examplebucket. If you want to delete one or more lifecycle rules, refer to Lifecycle rules based on the last modified time

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Delete the lifecycle rules. 
bucket.delete_bucket_lifecycle()

# An exception occurs when you query lifecycle rules again. 
try:
    lifecycle = bucket.get_bucket_lifecycle()
except oss2.exceptions.NoSuchLifecycle:
    print('lifecycle is not configured')

References

For the complete sample code for lifecycle rules, visit GitHub.
For more information about the API operation for configuring a lifecycle rule, see PutBucketLifecycle.
For more information about the API operation that you can call to query lifecycle rules, see GetBucketLifecycle.
For more information about the API operation for clearing lifecycle rules, see DeleteBucketLifecycle.