How to list files in a bucket using the Python SDK - Object Storage Service

This topic describes how to list objects in a bucket that has versioning enabled. You can list all objects, a specified number of objects, or objects that have a specified prefix.

Usage notes

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials using OSS SDK for Python 1.0.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.
To list objects, you must have the oss:ListObjectVersions permission. For more information, see Attach a custom policy to a RAM user.

List all object versions in a bucket

The following code shows how to list the version information of all objects, including delete markers, in a specified bucket:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region where the bucket is located, such as cn-hangzhou. Note: This parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)


# After you enable versioning for the bucket, you can call the list_object_versions operation to return information about different object versions.
# List the version information of all objects, including delete markers, in the bucket.
result = bucket.list_object_versions()

# List the version information of all objects.
next_key_marker = None
next_versionid_marker = None
while True:
    result = bucket.list_object_versions(key_marker=next_key_marker, versionid_marker=next_versionid_marker)

    # View the version information of the listed objects.
    for version_info in result.versions:
        print('version_info.versionid:', version_info.versionid)
        print('version_info.key:', version_info.key)
        print('version_info.is_latest:', version_info.is_latest)

    # View the version information of the listed delete markers.
    for del_maker_Info in result.delete_marker:
        print('del_maker.key:', del_maker_Info.key)
        print('del_maker.versionid:', del_maker_Info.versionid)
        print('del_maker.is_latest:', del_maker_Info.is_latest)

    is_truncated = result.is_truncated

    # Check whether the list is complete. If the list is not complete, continue to list the remaining objects. If the list is complete, exit the loop.
    if is_truncated:
        next_key_marker = result.next_key_marker
        next_versionid_marker = result.next_versionid_marker
    else:
        break

List object versions with a specified prefix

The following code shows how to list the version information of objects that have a specified prefix:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region where the bucket is located, such as cn-hangzhou. Note: This parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# After you enable versioning for the bucket, you can call the list_object_versions operation to return information about different object versions.
# List the version information of all objects, including delete markers, in the bucket.
result = bucket.list_object_versions()

# Specify that you want to list the version information of objects whose names start with the "test-" prefix.
prefix = 'test-'
next_key_marker = None
next_versionid_marker = None
while True:
    result = bucket.list_object_versions(prefix=prefix, key_marker=next_key_marker, versionid_marker=next_versionid_marker)

    # View the version information of the listed objects.
    for version_info in result.versions:
        print('version_info.versionid:', version_info.versionid)
        print('version_info.key:', version_info.key)
        print('version_info.is_latest:', version_info.is_latest)

    # View the version information of the listed delete markers.
    for del_maker_Info in result.delete_marker:
        print('del_maker.key:', del_maker_Info.key)
        print('del_maker.versionid:', del_maker_Info.versionid)
        print('del_maker.is_latest:', del_maker_Info.is_latest)

    is_truncated = result.is_truncated

    # Check whether the list is complete. If the list is not complete, continue to list the remaining objects. If the list is complete, exit the loop.
    if is_truncated:
        next_key_marker = result.next_key_marker
        next_versionid_marker = result.next_versionid_marker
    else:
        break

List a specified number of object versions

The following code shows how to list the version information of a specified number of objects:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region where the bucket is located, such as cn-hangzhou. Note: This parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# After you enable versioning for the bucket, you can call the list_object_versions operation to return information about different object versions.
# List the version information of all objects, including delete markers, in the bucket.
result = bucket.list_object_versions()

# Specify that a maximum of 200 results can be returned.
max_keys = 200

result = bucket.list_object_versions(max_keys=max_keys)

# View the version information of the listed objects.
for version_info in result.versions:
    print('version_info.versionid:', version_info.versionid)
    print('version_info.key:', version_info.key)
    print('version_info.is_latest:', version_info.is_latest)

# View the version information of the listed object delete markers.
for del_maker_Info in result.delete_marker:
    print('del_maker.key:', del_maker_Info.key)
    print('del_maker.versionid:', del_maker_Info.versionid)
    print('del_maker.is_latest:', del_maker_Info.is_latest)

# Check whether the list is truncated.
# You specified that a maximum of 200 results can be returned. If the number of objects in the bucket is greater than 200, the list is truncated and is_truncated is True. If the number of objects is less than 200, the list is not truncated and is_truncated is False.
print('is truncated', result.is_truncated)

Folder feature

OSS does not have a native folder structure. All elements are stored as objects. You can create a folder by creating a zero-byte object whose name ends with a forward slash (/). This object can be uploaded and downloaded. The OSS console displays objects whose names end with a forward slash (/) as folders.

You can use the `delimiter` and `prefix` parameters to simulate folders:

If you set `prefix` to a folder name, all objects whose names start with this prefix are listed. All files and subfolders in the folder are displayed as objects.
If you set `prefix` and also set `delimiter` to a forward slash (/), only the files and subfolders in that folder are listed. The subfolders are displayed as `CommonPrefixes`. The files and folders within those subfolders are not displayed.

Assume that a bucket named `examplebucket` contains the objects oss.jpg, fun/test.jpg, fun/movie/001.avi, and fun/movie/007.txt, and uses a forward slash (/) as the folder separator. The folder structure is as follows:

examplebucket           
 └── oss.jpg
 └── fun               
      └── test.jpg
      └── movie
           └── 001.avi
           └── 007.txt

The following examples show how to list objects by simulating folders.

List object versions in the root directory

The following code shows how to list the version information of objects in the root directory:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region where the bucket is located, such as cn-hangzhou. Note: This parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# After you enable versioning for the bucket, you can call the list_object_versions operation to return information about different object versions.
# List the version information of all objects, including delete markers, in the bucket.
result = bucket.list_object_versions()

# Set delimiter to a forward slash (/).
delimiter = "/"
next_key_marker = None
next_versionid_marker = None
while True:
    result = bucket.list_object_versions(delimiter=delimiter, key_marker=next_key_marker, versionid_marker=next_versionid_marker)

    # View the version information of the listed objects.
    for version_info in result.versions:
        print('version_info.versionid:', version_info.versionid)
        print('version_info.key:', version_info.key)
        print('version_info.is_latest:', version_info.is_latest)

    # View the version information of the listed delete markers.
    for del_maker_Info in result.delete_marker:
        print('del_maker.key:', del_maker_Info.key)
        print('del_maker.versionid:', del_maker_Info.versionid)
        print('del_maker.is_latest:', del_maker_Info.is_latest)

    # View the names of the directories that end with a forward slash (/).
    for common_prefix in result.common_prefix:
        print("common_prefix:", common_prefix)

    is_truncated = result.is_truncated

    # Check whether the list is complete. If the list is not complete, continue to list the remaining objects. If the list is complete, exit the loop.
    if is_truncated:
        next_key_marker = result.next_key_marker
        next_versionid_marker = result.next_versionid_marker
    else:
        break

Result:

('version_info.versionid:', 'CAEQEhiBgMCw8Y7FqBciIGIzMDE3MTEzOWRiMDRmZmFhMmRlMjljZWI0MWU4****')
('version_info.key:', 'oss.jpg')
('version_info.is_latest:', True)
('common_prefix:', 'fun/')

List files and subfolders in a folder

The following code shows how to list the files and subfolders in a specified folder:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region where the bucket is located, such as cn-hangzhou. Note: This parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# After you enable versioning for the bucket, you can call the list_object_versions operation to return information about different object versions.
# List the version information of all objects, including delete markers, in the bucket.
result = bucket.list_object_versions()

# Set delimiter to a forward slash (/) and prefix to fun/.
prefix = "fun/"
delimiter = "/"
next_key_marker = None
next_versionid_marker = None
while True:
    result = bucket.list_object_versions(prefix=prefix, delimiter=delimiter, key_marker=next_key_marker, versionid_marker=next_versionid_marker)

    # View the version information of the listed objects.
    for version_info in result.versions:
        print('version_info.versionid:', version_info.versionid)
        print('version_info.key:', version_info.key)
        print('version_info.is_latest:', version_info.is_latest)

    # View the version information of the listed delete markers.
    for del_maker_Info in result.delete_marker:
        print('del_maker.key:', del_maker_Info.key)
        print('del_maker.versionid:', del_maker_Info.versionid)
        print('del_maker.is_latest:', del_maker_Info.is_latest)

    # View the names of the folders that end with a forward slash (/).
    for common_prefix in result.common_prefix:
        print("common_prefix:", common_prefix)

    is_truncated = result.is_truncated

    # Check whether the list is complete. If the list is not complete, continue to list the remaining objects. If the list is complete, exit the loop.
    if is_truncated:
        next_key_marker = result.next_key_marker
        next_versionid_marker = result.next_versionid_marker
    else:
        break

Result:

('version_info.versionid:', 'CAEQFRiBgMCh9JDkrxciIGE3OTNkYzFhYTc2YzQzOTQ4Y2MzYjg2YjQ4ODg*****')
('version_info.key:', 'fun/test.jpg')
('version_info.is_latest:', True)
('commonPrefix:', 'fun/movie/')

References

For more information about the API operation for listing objects, see ListObjectVersions (GetBucketVersions).