How do I list all objects in a bucket, objects whose names contain a specific prefix, and objects and subdirectories in a specific directory of a bucket? - Object Storage Service

This topic describes how to list all objects, objects whose names contain a specific prefix, and objects and subdirectories in a specific directory of an Object Storage Service (OSS) bucket.

Usage notes

Only OSS SDK for Python 2.12.0 and later support the GetBucketV2 (ListObjectsV2) operation.
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions, endpoints and open ports.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.
To list objects, you must have the oss:ListObjects permission. For more information, see Attach a custom policy to a RAM user.

Background Information

You can call the GetBucket (ListObjects) or GetBucketV2 (ListObjectsV2) operation to list up to 1,000 objects in a bucket at a time. You can specify list operation parameters to list objects based on your business requirements. For example, you can list all the objects after a specific position, list all objects and subdirectories in a specific directory, and list more than 1,000 objects by page. The GetBucket (ListObjects) and GetBucketV2 (ListObjectsV2) operations have some differences:

When you call the GetBucket (ListObjects) operation to list objects, the object owner information is included in the response.
When you call the GetBucketV2 (ListObjectsV2) operation to list objects, you can configure the fetchOwner parameter to specify whether to include the object owner information in the response.
Note
To list objects in a versioning-enabled bucket, we recommend that you call the GetBucketV2 (ListObjectsV2) operation.

The following tables describe the parameters that you can configure when you call the GetBucket (ListObjects) or GetBucketV2 (ListObjectsV2) operation to list objects.

GetBucket (ListObjects)

The following table describes the parameters that you can configure when you call the GetBucket (ListObjects) operation to list objects.

Parameter	Description
prefix	The prefix in the names of objects that you want to list.
delimiter	The character used to group the objects that you want to list by name.
marker	The position from which the list operation starts.

GetBucketV2 (ListObjectsV2)

The following table describes the parameters that you can configure when you call the GetBucketV2 (ListObjectsV2) operation to list objects.

Parameter	Description
prefix	The prefix in the names of objects that you want to list.
delimiter	The character used to group the objects that you want to list by name.
startAfter	The position from which the list operation starts.
fetchOwner	Specifies whether to include the owner information in the response. Valid values: true false

List a specific number of objects

GetBucket (ListObjects)

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list 10 objects in a bucket:

Note

The ListObjects operation is encapsulated in ObjectIterator for your convenience.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from itertools import islice

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List 10 objects in the bucket. 
for b in islice(oss2.ObjectIterator(bucket), 10):
    print(b.key)

GetBucketV2 (ListObjectsV2)

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list 10 objects in a bucket.

Note

The ListObjectsV2 operation is encapsulated in ObjectIteratorV2 for your convenience.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from itertools import islice
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List 10 objects in the bucket. 
for obj in islice(oss2.ObjectIteratorV2(bucket), 10):
    print(obj.key)

List all objects in a bucket

GetBucket (ListObjects)

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list all objects in a bucket:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects in the bucket. 
for obj in oss2.ObjectIterator(bucket):
    print(obj.key)

GetBucketV2 (ListObjectsV2)

List all objects in a bucket and do not include the object owner information in the response

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list all objects in a bucket and do not include the object owner information in the response:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects in the bucket. 
for obj in oss2.ObjectIteratorV2(bucket):
    print(obj.key)

List all objects in a bucket and include the object owner information in the response

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list all objects in a bucket and specify the fetchOwner parameter to include the object owner information in the response:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects in the bucket and specify the fetchOwner parameter to include the object owner information in the response. 
for obj in oss2.ObjectIteratorV2(bucket, fetch_owner=True):
    print(obj.key)
    print('file owner display name: ' + obj.owner.display_name)
    print('file owner id: ' + obj.owner.id)

List objects whose names contain a specific prefix

In the following examples, four objects are stored in a bucket: oss.jpg, fun/test.jpg, fun/movie/001.avi, and fun/movie/007.avi. A forward slash (/) is used as the directory delimiter.

GetBucket (ListObjects)

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list objects whose names contain a specific prefix:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects in the fun directory and its subdirectories. 
for obj in oss2.ObjectIterator(bucket, prefix='fun/'):
    print(obj.key)

GetBucketV2 (ListObjectsV2)

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list objects whose names contain a specific prefix:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects in the fun directory and its subdirectories. 
for obj in oss2.ObjectIteratorV2(bucket, prefix='fun/'):
    print(obj.key)

List all objects from a specific start position

GetBucket (ListObjects)

You can configure the marker parameter to specify the position from which the list operation starts. All objects whose names are alphabetically after the value of marker are returned. In the following example, four objects are stored in a bucket: x1.txt, x2.txt, z1.txt, and z2.txt.

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list all objects from a specific start position:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List all objects whose names are alphabetically after the value of marker. The object whose name is the same as the value of marker is not returned. 
for obj in oss2.ObjectIterator(bucket, marker="x2.txt"):
    print(obj.key)

GetBucketV2 (ListObjectsV2)

You can configure the start_after parameter to specify the position from which the list operation starts. All objects whose names are alphabetically after the value of start_after are returned. In the following example, four objects are stored in a bucket: x1.txt, x2.txt, z1.txt, and z2.txt.

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list all objects from a specific start position:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())
# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
# Specify the name of the bucket. Example: examplebucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')

# List all objects whose names are alphabetically after the value of marker. The object whose name is the same as the value of start_after is not returned. 
for obj in oss2.ObjectIteratorV2(bucket, start_after="x2.txt"):
    print(obj.key)

List objects and subdirectories in a specific directory

OSS uses a flat structure instead of a hierarchical structure to store objects. A directory is a zero-byte object whose name ends with a forward slash (/). You can upload and download this directory. By default, objects whose names end with a forward slash (/) are displayed as directories in the OSS console.

You can specify the delimiter and prefix parameters to list objects by directory.

If you set the prefix to the name of a directory, objects and subdirectories whose names contain the prefix are listed.
If you specify a prefix and specify a forward slash (/) as the delimiter, only the objects and subdirectories in the directory are listed. The objects and directories in the subdirectories are not listed.

GetBucket (ListObjects)

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list objects and subdirectories in a specific directory:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List objects and subdirectories in the fun directory. Objects in the subdirectories are not listed. 
for obj in oss2.ObjectIterator(bucket, prefix='fun/', delimiter ='/'):
    # Use is_prefix to determine whether obj is a directory. (Configuration of delimiter & prefix is required to list objects in a folder and determine whether obj is a directory.)
    if obj.is_prefix():  # Specify the operation to perform if obj is determined as a directory. 
        print('directory: ' + obj.key)
    else:                # Specify the operation to perform if obj is determined as an object. 
        print('file: ' + obj.key)

GetBucketV2 (ListObjectsV2)

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list objects and subdirectories in a specific directory:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# List objects and subdirectories in the fun directory. Objects in the subdirectories are not listed. If you do not need the object owner information, do not configure the fetch_owner parameter. 
for obj in oss2.ObjectIteratorV2(bucket, prefix = 'fun/', delimiter = '/', start_after='fun/', fetch_owner=True):
    # Use is_prefix to determine whether obj is a directory. 
    if obj.is_prefix():  # Specify the operation to perform if obj is determined as a directory. 
        print('directory: ' + obj.key)
    else:                # Specify the operation to perform if obj is determined as an object. 
        print('file: ' + obj.key)
        print('file owner display name: ' + obj.owner.display_name)
        print('file owner id: ' + obj.owner.id)

List the sizes of objects in a specific directory

GetBucket (ListObjects)

The following sample code provides an example on how to call the GetBucket (ListObjects) operation to list the sizes of objects in a specific directory:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def CalculateFolderLength(bucket, folder):
    length = 0
    for obj in oss2.ObjectIterator(bucket, prefix=folder):
        length += obj.size
    return length
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

for obj in oss2.ObjectIterator(bucket, delimiter='/'):
    if obj.is_prefix():  # Specify the operation to perform if obj is determined as a directory. 
        length = CalculateFolderLength(bucket, obj.key)
        print('directory: ' + obj.key + '  length:' + str(length) + "Byte.")
    else: # Specify the operation to perform if obj is determined as an object. 
        print('file:' + obj.key + '  length:' + str(obj.size) + "Byte.")

GetBucketV2 (ListObjectsV2)

The following sample code provides an example on how to call the GetBucketV2 (ListObjectsV2) operation to list the sizes of objects in a specific directory:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def CalculateFolderLength(bucket, folder):
    length = 0
    for obj in oss2.ObjectIteratorV2(bucket, prefix=folder):
        length += obj.size
    return length
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)


for obj in oss2.ObjectIteratorV2(bucket, delimiter='/'):
    if obj.is_prefix():  # Specify the operation to perform if obj is determined as a directory. 
        length = CalculateFolderLength(bucket, obj.key)
        print('directory: ' + obj.key + '  length:' + str(length) + "Byte.")
    else: # Specify the operation to perform if obj is determined as a directory. 
        print('file:' + obj.key + '  length:' + str(obj.size) + "Byte.")

References

For the complete sample code that is used to list objects, visit GitHub.
For more information about the API operations that you can call to list objects, see ListObjects (GetBucket) and ListObjectsV2 (GetBucketV2).