All Products
Search
Document Center

Object Storage Service:Bucket inventory

Last Updated:Oct 08, 2024

You can use the bucket inventory feature to export information about specific objects in a bucket, such as the number, size, storage class, and encryption status of the objects. To list a large number of objects, we recommend that you use the bucket inventory feature instead of calling the GetBucket (ListObjects) operation.

Warning

To maintain OSS-HDFS availability and prevent data contamination, do not set Inventory Path to .dlsdata/ when you create an inventory for a bucket for which OSS-HDFS is enabled.

Billing rules

  • You are charged when you use the bucket inventory feature. However, you are charged only the storage fees for inventory lists, operation calling fees for the PutBucketInventory, GetBucketInventory, ListBucketInventory, and DeleteBucketInventory operations, and traffic and request fees for accessing the inventory lists during the public preview.

  • You are charged for the storage of the inventory lists. Object Storage Service (OSS) generates inventory lists based on the inventory. To prevent unnecessary costs, delete inventory lists that you no longer need.

Limits

  • You can configure up to 1,000 inventories for a bucket by using OSS SDKs or ossutil. You can configure up to 10 inventories for a bucket by using the OSS console.

  • The bucket for which you want to configure an inventory can be different from the bucket in which you want to store the generated inventory lists. However, the two buckets must belong to the same Alibaba Cloud account and be located in the same region.

Permissions

When you use the bucket inventory feature, you must configure an inventory for a bucket. Then, OSS assumes the RAM role that you created to write the generated inventory lists to the inventory storage bucket.

You can perform the following steps to grant a RAM user role-related permissions and the permissions to configure inventories, create a RAM role, and grant permissions to the RAM role. Then, you can create an inventory. For more information, see Create an inventory.

Important

High risks may arise when you grant a RAM user role-related permissions, such as the CreateRole and GetRoles permissions. We recommend that you create a RAM role and grant permissions to the RAM role by using the Alibaba Cloud account to which the RAM user belongs. Then, the RAM user can assume the RAM role that is created by the Alibaba Cloud account.

  • Grant a RAM user role-related permissions and the permissions to configure inventories

    Perform the following steps to grant a RAM user role-related permissions and the permissions to configure inventories:

    1. Create the following custom policy on the JSON tab. For more information, see Create custom policies.

      Note

      If you access OSS by using the OSS console, you must add oss:ListBuckets to the Action element in the policy. If you access OSS by using OSS SDKs or ossutil, you do not need to add oss:ListBuckets to the Action element in the policy.

      {
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "oss:PutBucketInventory",
                      "oss:GetBucketInventory",
                      "oss:DeleteBucketInventory",
                      "oss:ListBuckets",
                      "ram:CreateRole",
                      "ram:AttachPolicyToRole",
                      "ram:GetRole",
                      "ram:ListPoliciesForRole"
                  ],
                  "Resource": "*"
              }
          ],
          "Version": "1"
      }
      Important

      The AliyunOSSFullAccess policy allows you to perform all operations on OSS resources. If the AliyunOSSFullAccess policy is attached to a RAM user that you created in your Alibaba Cloud account, you need to only attach the following policy to the RAM user to grant role-related permissions to the RAM user.

      {
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "ram:CreateRole",
                      "ram:AttachPolicyToRole",
                      "ram:GetRole",
                      "ram:ListPoliciesForRole"
                  ],
                  "Resource": "*"
              }
          ],
          "Version": "1"
      }
    2. Attach the created custom policy to the RAM user. For more information, see Grant permissions to a RAM user.

      After the RAM user is granted the required permissions, the RAM user can create a RAM role and grant permissions to the RAM role. For more information, see What to do next.

  • Create a RAM role and grant permissions to the RAM role

    Important

    If you want to use a key that is managed by Key Management Service (KMS) to encrypt inventory lists, you must attach the AliyunKMSFullAccess policy to the created RAM role to grant the RAM role the permissions to manage KMS.

    • Grant permissions to the AliyunOSSRole RAM role that is automatically created

      When you configure an inventory in the OSS console, the RAM console automatically creates the AliyunOSSRole RAM role. By default, the AliyunOSSRole RAM role has the permissions to write inventory lists to the inventory storage bucket.

      Important

      If you use the AliyunOSSRole RAM role, you do not need to grant permissions to the RAM role. However, security risks may arise because the RAM role has the permissions to manage OSS by default. You can create a custom RAM role and grant the least permissions to the role to comply with the principle of least privilege.

    • Create a custom RAM role and grant permissions to the role

      If you want OSS to assume a custom RAM role to write inventory lists to the inventory storage bucket, perform the following steps to create a custom RAM role and attach a policy to the role:

      1. Create a normal service role.

        When you create a normal service role, select OSS for the Select Trusted Service parameter. For information about how to configure other parameters, see Create a RAM role for a trusted Alibaba Cloud service.

      2. Create a custom policy on the Visual editor tab.

        When you create a custom policy, configure the parameters. The following table describes the parameters. For information about how to configure other parameters, see Create custom policies.

        Parameter

        Description

        Effect

        Select Allow.

        Service

        Select Object Storage Service.

        Action

        Select Select action(s) and then select oss:PutObject in Write actions.

        Resource

        Select All resource(s)(*).

        The following code shows the policy in JSON format:

        {
          "Version": "1",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": "oss:PutObject",
              "Resource": "*"
            }
          ]
        }
      3. Attach the custom policy to the RAM role.

        For more information, see Grant permissions to a RAM role.

Procedure

Use the OSS console

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

  3. In the left-side navigation tree, choose Data Management > Bucket Inventory.

  4. On the Bucket Inventory page, click Create Inventory.

  5. In the Create Inventory panel, configure the parameters. The following table describes the parameters.

    Parameter

    Description

    Status

    The status of the inventory. Valid values: Enabled and Disabled.

    Rule Name

    The name of the inventory. The name can contain only lowercase letters, digits, and hyphens (-), but cannot start or end with a hyphen (-).

    Inventory Storage Bucket

    The bucket in which the generated inventory lists are stored.

    The bucket for which you want to configure an inventory can be different from the bucket in which you want to store the generated inventory lists. However, the two buckets must belong to the same Alibaba Cloud account and be located in the same region.

    Inventory Path

    The directory in which the generated inventory lists are stored.

    • If you want to store the inventory lists in the root directory of the inventory storage bucket, leave the parameter empty.

    • Otherwise, set the parameter to the full path of a directory, excluding the name of the inventory storage bucket.

      For example, if you want to store the inventory lists in the exampledir1 path of the inventory storage bucket named examplebucket, set the parameter to exampledir1. If you want to store the inventory lists in the exampledir1/exampledir2 path of the inventory storage bucket named examplebucket, set the parameter to exampledir1/exampledir2.

    Note

    If the path that you specify does not exist, OSS creates the path.

    Frequency

    The frequency at which inventory lists are generated. Valid values: Weekly and Daily.

    • If fewer than 10 billion objects are stored in a bucket, you can specify that inventory lists are exported on a daily or weekly basis based on your business requirements.

    • If more than 10 billion objects are stored in a bucket, you can specify that inventory lists are exported on a weekly basis.

    Encryption Method

    Specifies whether to encrypt inventory lists.

    • None: Inventory lists are not encrypted.

    • AES256: Inventory lists are encrypted by using AES256.

    • KMS: Inventory lists are encrypted by using a customer master key (CMK) managed by KMS.

      To use a CMK to encrypt inventory lists, you must create a CMK in KMS in the same region in which the inventory storage bucket is located. For more information, see Create a CMK.

      Note

      You are charged API operation calling fees when you use CMKs to encrypt or decrypt data.

    Optional Fields

    The object information that you want to include in the inventory lists that you want to export. Valid values: Object Size, Storage Class, Last Modified Date, ETag, Status of Multipart Upload, Encryption Status, Object ACL, Number of Tags, Object Type, and CRC-64.

    Object Prefix

    The prefix based on which objects are scanned.

    • If you want OSS to scan all objects in the bucket, leave the parameter empty.

    • To scan all objects in a specific path of the bucket, set this parameter to the full path that does not contain the bucket name.

      For example, if you want to scan all objects in the exampledir1 path of the examplebucket bucket, set this parameter to exampledir1/. To scan all objects in the exampledir1/exampledir2 path of the examplebucket bucket, set this parameter to exampledir1/exampledir2/.

    Note

    If no objects in the bucket match the specified prefix, no inventory lists are generated.

    Advanced Filtering

    If you want to filter exported objects based on conditions, such as the size and storage class of the objects, turn on Advanced Filtering. The following filtering options are supported:

    Important

    You can configure filtering options only for buckets that are located in the China (Qingdao), China (Hohhot), Germany (Frankfurt), and Australia (Sydney, closing Down) regions.

    • Time Range: You can specify Start Date and End Date of the last modified time of the objects that you want to include in the inventory list. The values of Start Date and End Date are accurate to seconds.

    • Object Size Range: Specify the minimum size and maximum size of the objects that you want to include in the inventory list.

      Important

      When you specify the range of object size, make sure that the minimum size and maximum size are greater than 0 B and the maximum size is less than or equal to 48.8 TB.

    • Storage Class: Specify the storage class of the objects that you want to export. Valid values: Standard, IA, Archive, Cold Archive and Deep Cold Archive.

    Object Versions

    The object versions for which you want to configure an inventory.

    If versioning is enabled for the bucket, you can select Current Version or All Versions. For more information, see Overview.

    By default, inventory lists are generated for all objects in a bucket for which versioning is disabled.

  6. Select I understand the terms and agree to authorize Alibaba Cloud OSS to access the resources in my buckets., and click OK.

    A long period of time may be required to generate inventory lists for a large number of objects. You can use two methods to check whether an inventory list is generated. For more information, see How do I determine whether inventory lists are generated?

Use OSS SDKs

The following sample code provides examples on how to configure an inventory for a bucket by using OSS SDKs for common programming languages. For more information about how to configure an inventory for a bucket by using OSS SDKs for other programming languages, see Overview.

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.*;
import java.util.ArrayList;
import java.util.List;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the name of the bucket in which you want to store the generated inventory lists. 
        String destBucketName ="yourDestinationBucketName";
        // Specify the account ID granted by the bucket owner. 
        String accountId ="yourDestinationBucketAccountId";
        // Specify the name of the RAM role that is granted the permissions to read all objects in the bucket for which you want to configure the inventory and the permissions to write data to the bucket in which you want to store the generated inventory lists. 
        String roleArn ="yourDestinationBucketRoleArn";

        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, credentialsProvider);

        try {
            // Create an inventory. 
            InventoryConfiguration inventoryConfiguration = new InventoryConfiguration();

            // Specify the inventory name. 
            String inventoryId = "testid";
            inventoryConfiguration.setInventoryId(inventoryId);

            // Specify the object attributes that are included in inventory lists. 
            List<String> fields = new ArrayList<String>();
            fields.add(InventoryOptionalFields.Size);
            fields.add(InventoryOptionalFields.LastModifiedDate);
            fields.add(InventoryOptionalFields.IsMultipartUploaded);
            fields.add(InventoryOptionalFields.StorageClass);
            fields.add(InventoryOptionalFields.ETag);
            fields.add(InventoryOptionalFields.EncryptionStatus);
            inventoryConfiguration.setOptionalFields(fields);

            // Specify whether to generate inventory lists on a daily or weekly basis. The following code provides an example on how to generate the inventory lists on a weekly basis. Weekly indicates that the inventory lists are generated once a week and Daily indicates that the inventory lists are generated once a day. 
            inventoryConfiguration.setSchedule(new InventorySchedule().withFrequency(InventoryFrequency.Weekly));

            // Specify that the inventory lists include only the current version of objects. If you set the InventoryIncludedObjectVersions parameter to All, all versions of objects are included in the inventory lists. This configuration takes effect only when you enable versioning for the bucket. 
            inventoryConfiguration.setIncludedObjectVersions(InventoryIncludedObjectVersions.Current);

            // Specify whether the inventory is enabled. Valid values: true and false. Set the value to true to enable the inventory. Set the value to false to disable the inventory. 
            inventoryConfiguration.setEnabled(true);

            // Specify the rule used to filter the objects to include in the inventory lists. The following code provides an example on how to filter the objects by prefix. 
            InventoryFilter inventoryFilter = new InventoryFilter().withPrefix("obj-prefix");
            inventoryConfiguration.setInventoryFilter(inventoryFilter);

            // Specify the destination bucket in which you want to store the generated inventory lists. 
            InventoryOSSBucketDestination ossInvDest = new InventoryOSSBucketDestination();
            // Specify the prefix of the path in which you want to store the generated inventory lists. 
            ossInvDest.setPrefix("destination-prefix");
            // Specify the format of the inventory lists. 
            ossInvDest.setFormat(InventoryFormat.CSV);
            // Specify the ID of the account to which the destination bucket belongs. 
            ossInvDest.setAccountId(accountId);
            // Specify the role ARN of the destination bucket. 
            ossInvDest.setRoleArn(roleArn);
            // Specify the name of the destination bucket. 
            ossInvDest.setBucket(destBucketName);

            // The following code provides an example on how to encrypt the inventory lists by using customer master keys (CMKs) hosted in Key Management System (KMS). 
            // InventoryEncryption inventoryEncryption = new InventoryEncryption();
            // InventoryServerSideEncryptionKMS serverSideKmsEncryption = new InventoryServerSideEncryptionKMS().withKeyId("test-kms-id");
            // inventoryEncryption.setServerSideKmsEncryption(serverSideKmsEncryption);
            // ossInvDest.setEncryption(inventoryEncryption);

            // The following code provides an example on how to encrypt the inventory lists on the OSS server. 
            // InventoryEncryption inventoryEncryption = new InventoryEncryption();
            // inventoryEncryption.setServerSideOssEncryption(new InventoryServerSideEncryptionOSS());
            // ossInvDest.setEncryption(inventoryEncryption);

            // Specify the destination for the generated inventory lists. 
            InventoryDestination destination = new InventoryDestination();
            destination.setOssBucketDestination(ossInvDest);
            inventoryConfiguration.setDestination(destination);

            // Configure the inventory for the bucket. 
            ossClient.setBucketInventoryConfiguration(bucketName, inventoryConfiguration);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}
const OSS = require('ali-oss');

const client = new OSS({
  // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
  region: 'yourregion',
  // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
  accessKeyId: process.env.OSS_ACCESS_KEY_ID,
  accessKeySecret: process.env.OSS_ACCESS_KEY_SECRET,
  // Specify the name of the bucket. 
  bucket: 'yourbucketname'
});

const inventory = {
  // Specify the ID of the inventory. 
  id: 'default', 
  // Specify whether to enable the bucket inventory feature for the bucket. Valid values: true and false. 
  isEnabled: false, 
  // (Optional) Specify the condition that is used to filter the objects in inventory lists. The following sample code provides an example on how to filter the objects by prefix: 
  prefix: 'ttt',
  OSSBucketDestination: {
     // Specify the format of inventory lists. 
    format: 'CSV',
   // Specify the ID of the Alibaba Cloud account to which the inventory storage bucket belongs. 
    accountId: '<Your AccountId>', 
   // Specify the name of the RAM role that is used to access the inventory storage bucket. 
    rolename: 'AliyunOSSRole',
    // Specify the name of the inventory storage bucket. 
    bucket: '<Your BucketName>',
    // (Optional) Specify the prefix of the path in which you want to store the generated inventory lists. 
    prefix: '<Your Prefix>',
    // The following sample code provides an example on how to encrypt the inventory lists by using OSS-managed keys (SSE-OSS): 
    //encryption: {'SSE-OSS': ''},
    // The following sample code provides an example on how to encrypt the inventory lists by using CMKs managed by KMS (SSE-KMS): 
           /*
            encryption: {
      'SSE-KMS': {
        keyId: 'test-kms-id',
      };, 
    */
  },
  // Specify the frequency at which inventory lists are generated. A value of Weekly specifies that inventory lists are generated on a weekly basis. A value of Daily specifies that inventory lists are generated on a daily basis. 
  frequency: 'Daily', 
  // Specify that all versions of the objects are included in inventory lists. If includedObjectVersions is set to Current, only the current versions of the objects are included in inventory lists. 
  includedObjectVersions: 'All', 
  optionalFields: {
    // (Optional) Specify the object attributes that you want to include in inventory lists. 
    field: ["Size", "LastModifiedDate", "ETag", "StorageClass", "IsMultipartUploaded", "EncryptionStatus"]
  },
}

async function putInventory(){
  // Specify the name of the bucket for which you want to configure an inventory. 
  const bucket = '<Your BucketName>'; 
        try {
    await client.putBucketInventory(bucket, inventory);
    console.log('An inventory is configured.')
  } catch(err) {
    console.log('Failed to configure an inventory: ', err);
  }
}

putInventory()
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import (InventoryConfiguration,
                         InventoryFilter,
                         InventorySchedule,
                         InventoryDestination,
                         InventoryBucketDestination,
                         INVENTORY_INCLUDED_OBJECT_VERSIONS_CURRENT,
                         INVENTORY_FREQUENCY_DAILY,
                         INVENTORY_FORMAT_CSV,
                         FIELD_SIZE,
                         FIELD_LAST_MODIFIED_DATE,
                         FIELD_STORAG_CLASS,
                         FIELD_ETAG,
                         FIELD_IS_MULTIPART_UPLOADED,
                         FIELD_ENCRYPTION_STATUS)

# Obtain access credentials from the environment variables. Before you run the sample code, make sure that you have configured environment variables OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET. 
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())
# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
# Specify the name of the bucket. Example: examplebucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')

# Specify the ID of the account to which the bucket owner grants permissions to perform the operation. Example: 1283641033516515.
account_id = 'yourtBucketDestinationAccountId'
# Specify the name of the RAM role that is granted permissions to read all objects in the source bucket for which you want to configure the inventory and permissions to write data to the destination bucket in which you want to store the generated inventory lists. Example: acs:ram::1283641033516515:role/AliyunOSSRole.
role_arn = 'yourBucketDestinationRoleArn'
# Specify the name of the bucket in which you want to store the generated inventory lists. 
dest_bucket_name = 'yourDestinationBucketName'

# Specify the inventory name. 
inventory_id = "inventory1"

# Specify the object attributes that are included in the inventory lists. 
optional_fields = [FIELD_SIZE, FIELD_LAST_MODIFIED_DATE, FIELD_STORAG_CLASS,
                   FIELD_ETAG, FIELD_IS_MULTIPART_UPLOADED, FIELD_ENCRYPTION_STATUS]

# Create the bucket in which you want to store the generated inventory lists. 
bucket_destination = InventoryBucketDestination(
    # Specify the ID of the account to which the destination bucket belongs. 
    account_id=account_id,
    # Specify the ARN of the RAM role that you want to use to access the destination bucket. 
    role_arn=role_arn,
    # Specify the name of the destination bucket. 
    bucket=dest_bucket_name,
    # Specify the format of the inventory lists. 
    inventory_format=INVENTORY_FORMAT_CSV,
    # Specify the prefix of the path in which you want to store the generated inventory lists. 
    prefix='destination-prefix',
    # The following code provides an example on how to encrypt inventory lists by using customer master keys (CMKs) that are managed by Key Management Service (KMS). 
    # sse_kms_encryption=InventoryServerSideEncryptionKMS("test-kms-id"),
    # The following code provides an example on how to encrypt the inventory lists on the OSS server. 
    # sse_oss_encryption=InventoryServerSideEncryptionOSS()
)

# Create an inventory. 
inventory_configuration = InventoryConfiguration(
    # Specify the ID of the inventory. 
    inventory_id=inventory_id,
    # Specify whether to enable the inventory for the bucket. Valid values: true and false. 
    is_enabled=True,
    # Specify whether to generate inventory lists on a daily or weekly basis. The following code provides an example on how to generate inventory lists on a daily basis. A value of Weekly indicates that inventory lists are generated on a weekly basis. A value of Daily indicates that inventory lists are generated on a daily basis. 
    inventory_schedule=InventorySchedule(frequency=INVENTORY_FREQUENCY_DAILY),
    # Specify that inventory lists include only the current versions of objects. If you set the INVENTORY_INCLUDED_OBJECT_VERSIONS parameter to ALL, all versions of objects are included in inventory lists. This inventory takes effect only when versioning is enabled for the source bucket. 
    included_object_versions=INVENTORY_INCLUDED_OBJECT_VERSIONS_CURRENT,
    # Specify the name prefix of objects that you want to include in inventory lists. 
    # inventory_filter=InventoryFilter(prefix="obj-prefix"),
    # Specify conditions to filter inventory lists. For example, the beginning of the time range during which the object was last modified is 1637883649. 
    inventory_filter=InventoryFilter(
        # Specify the prefix that is used to filter inventories. 
        "obj-prefix",
        # Specify the beginning of the time range during which the object was last modified. Unit: seconds. 
        1637883649,
        # Specify the end of the time range during which the object was last modified. Unit: seconds. 
        1638347592,
        # Specify the minimum size of the object. Unit: B. 
        1024,
        # Specify the maximum size of the object. Unit: B. 
        1048576,
        # Specify the storage classes of objects. 
        'Standard,IA'),
    # Specify the object attributes that you want to include in inventory lists. 
    optional_fields=optional_fields,    
    inventory_destination=InventoryDestination(bucket_destination=bucket_destination))

# Apply the inventory to the bucket. 
result = bucket.put_bucket_inventory_configuration(inventory_configuration)
print(result.status)
using Aliyun.OSS;
using Aliyun.OSS.Common;

// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket. 
var bucketName = "examplebucket";
// Specify the account ID granted by the bucket owner. 
var accountId ="yourDestinationBucketAccountId";
// Specify the name of the RAM role that is granted the permissions to read all objects in the bucket for which you want to configure the inventory and the permissions to write data to the bucket in which you want to store the generated inventory lists. 
var roleArn ="yourDestinationBucketRoleArn";
// Specify the name of the bucket in which you want to store the generated inventory lists. 
var destBucketName ="yourDestinationBucketName";
// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);
try
{
    // Create an inventory for the bucket. 
    var config = new InventoryConfiguration();
    // Specify the name of the inventory. 
    config.Id = "report1";
    // Specify whether to enable the inventory for the bucket. Valid values: true and false. If this parameter is set to true, the inventory is enabled. 
    config.IsEnabled = true;
    // Specify the rule that is used to filter the objects included in inventory lists. The following code provides an example on how to filter the objects by prefix. 
    config.Filter = new InventoryFilter("filterPrefix");
    // Configure the bucket in which you want to store the generated inventory lists. 
    config.Destination = new InventoryDestination();
    config.Destination.OSSBucketDestination = new InventoryOSSBucketDestination();
    // Specify the format of the inventory lists. 
    config.Destination.OSSBucketDestination.Format = InventoryFormat.CSV;
    // Specify the ID of the account to which the destination bucket belongs. 
    config.Destination.OSSBucketDestination.AccountId = accountId;
    // Specify the Alibaba Cloud Resource Name (ARN) of the RAM role that is used to access the destination bucket. 
    config.Destination.OSSBucketDestination.RoleArn = roleArn;
    // Specify the name of the bucket in which you want to store the generated inventory lists. 
    config.Destination.OSSBucketDestination.Bucket = destBucketName;
    // Specify the prefix of the path in which you want to store the generated inventory lists. 
    config.Destination.OSSBucketDestination.Prefix = "prefix1";
    
    // Specify whether to generate the inventory lists on a daily or weekly basis. The following code provides an example on how to generate the inventory lists on a weekly basis. A value of Weekly indicates that the inventory lists are generated on a weekly basis. A value of Daily indicates that the inventory lists are generated on a daily basis. 
    config.Schedule = new InventorySchedule(InventoryFrequency.Daily);
    // Specify that the inventory lists include only the current versions of objects. If you set the InventoryIncludedObjectVersions parameter to All, all versions of objects are included in the inventory lists. This configuration takes effect only when versioning is enabled for the bucket. 
    config.IncludedObjectVersions = InventoryIncludedObjectVersions.All;
    
    // Specify the object attributes that are included in the inventory lists. 
    config.OptionalFields.Add(InventoryOptionalField.Size);
    config.OptionalFields.Add(InventoryOptionalField.LastModifiedDate);
    config.OptionalFields.Add(InventoryOptionalField.StorageClass);
    config.OptionalFields.Add(InventoryOptionalField.IsMultipartUploaded);
    config.OptionalFields.Add(InventoryOptionalField.EncryptionStatus);
    config.OptionalFields.Add(InventoryOptionalField.ETag);
    var req = new SetBucketInventoryConfigurationRequest(bucketName, config);
    client.SetBucketInventoryConfiguration(req);
    Console.WriteLine("Set bucket:{0} InventoryConfiguration succeeded", bucketName);
}
catch (OssException ex)
{
    Console.WriteLine("Failed with error code: {0}; Error info: {1}. \nRequestID:{2}\tHostID:{3}",
        ex.ErrorCode, ex.Message, ex.RequestId, ex.HostId);
}
package main

import (
	"fmt"
	"github.com/aliyun/aliyun-oss-go-sdk/oss"
	"os"
)

func main() {
	/// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance. 
	// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
	client, err := oss.New("yourEndpoint", "", "", oss.SetCredentialsProvider(&provider))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	IsEnabled := true

	// The following code provides an example on how to encrypt the inventory lists by using customer master keys (CMKs) managed by Key Management Service (KMS). 
	//var invEncryption oss.InvEncryption
	//var invSseOss oss.InvSseOss
	//var invSseKms oss.InvSseKms
	//invSseKms.KmsId = "<yourKmsId>" // Specify the ID of CMK that is managed by KMS. 
	//invEncryption.SseOss = &invSseOss // Use OSS-managed keys (SSE-OSS) to encrypt the inventory lists. 
	//invEncryption.SseKms = &invSseKms // Use CMKs managed by KMS (SSE-KMS) to encrypt inventory lists. 

	invConfig := oss.InventoryConfiguration{
		// Specify the name of the inventory. The name must be globally unique in the current bucket. 
		Id: "yourInventoryId2",
		// Enable the inventory. 
		IsEnabled: &IsEnabled,

		// Specify the rule that is used to filter the objects included in inventories. The following code provides an example on how to filter the objects by prefix. 
		Prefix: "yourFilterPrefix",
		OSSBucketDestination: oss.OSSBucketDestination{
			// Specify the format of the exported inventory lists. 
			Format: "CSV",

			// Specify the ID of the account that is granted permissions by the bucket owner to perform the operation. Example: 109885487000****. 
			AccountId: "yourGrantAccountId",

			// Specify the name of the RAM role that is granted permissions by the bucket owner to perform the operation. Example: acs:ram::109885487000****:role/ram-test. 
			RoleArn: "yourRoleArn",

			// Specify the name of the bucket in which you want to store the generated inventory lists. 
			Bucket: "acs:oss:::" + "yourDestBucketName",

			// Specify the prefix of the path in which you want to store the generated inventory lists. 
			Prefix: "yourDestPrefix",

			// The following sample code provides an example on how to encrypt inventory lists. 
			//Encryption:     &invEncryption,
		},

		// Specify the frequency at which inventory lists are exported. 
		Frequency: "Daily",

		// Specify whether to include all versions of objects or only the current versions of objects in the inventory lists. 
		IncludedObjectVersions: "All",

		OptionalFields: oss.OptionalFields{
			// Specify the fields that are included in inventory lists. 
			Field: []string{
				"Size", "LastModifiedDate", "ETag", "StorageClass", "IsMultipartUploaded", "EncryptionStatus",
			},
		},
	}
	// Specify the name of the bucket for which you want to configure the inventory. 
	err = client.SetBucketInventory("yourBucketName", invConfig)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
}
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
            
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";

    /* Initialize resources, such as network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);

    InventoryConfiguration inventoryConf;
    /* Specify the name of the inventory. The name must be globally unique in the current bucket. */
    inventoryConf.setId("inventoryId");

    /* Specify whether to enable inventory for the bucket. Valid values: true and false. */
    inventoryConf.setIsEnabled(true);

    /* (Optional) Specify the prefix in the names of the objects. After you specify the prefix, information about the objects whose names contain the prefix is included in the inventory lists. */
    inventoryConf.setFilter(InventoryFilter("objectPrefix"));

    InventoryOSSBucketDestination dest;
    /* Specify the format of the exported inventory lists. */
    dest.setFormat(InventoryFormat::CSV);
    /* Specify the ID of the Alibaba Cloud account to which the bucket owner grants the permissions to perform the operation. */
    dest.setAccountId("10988548********");
    /* Specify the name of the RAM role to which the bucket owner grants permissions to perform the operation. */
    dest.setRoleArn("acs:ram::10988548********:role/inventory-test");
    /* Specify the bucket in which you want to store the generated inventory lists. */
    dest.setBucket("yourDstBucketName");
    /* Specify the prefix of the path in which you want to store the generated inventory lists. */
    dest.setPrefix("yourPrefix");
    /* (Optional) Specify the method that is used to encrypt inventory lists. Valid values: SSEOSS and SSEKMS. */
    //dest.setEncryption(InventoryEncryption(InventorySSEOSS()));
    //dest.setEncryption(InventoryEncryption(InventorySSEKMS("yourKmskeyId")));
    inventoryConf.setDestination(dest);

    /* Specify the time interval at which inventory lists are exported. Valid values: Daily and Weekly. */
    inventoryConf.setSchedule(InventoryFrequency::Daily);

    /* Specify whether to include all versions of objects or only the current versions of objects in the inventory lists. Valid values: All and Current. */
    inventoryConf.setIncludedObjectVersions(InventoryIncludedObjectVersions::All);

    /* (Optional) Specify the fields that are included in inventory lists based on your requirements. */
    InventoryOptionalFields field { 
        InventoryOptionalField::Size, InventoryOptionalField::LastModifiedDate, 
        InventoryOptionalField::ETag, InventoryOptionalField::StorageClass, 
        InventoryOptionalField::IsMultipartUploaded, InventoryOptionalField::EncryptionStatus
    };
    inventoryConf.setOptionalFields(field);

    /* Configure the inventory. */
    auto outcome = client.SetBucketInventoryConfiguration(
        SetBucketInventoryConfigurationRequest(BucketName, inventoryConf));

    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "Set Bucket Inventory fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release resources, such as network resources. */
    ShutdownSdk();
    return 0;
}

Use ossutil

For more information about how to use ossutil to configure an inventory for a bucket, see inventory.

Use the OSS API

If your business requires a high level of customization, you can directly call RESTful APIs. To directly call an API, you must include the signature calculation in your code. For more information, see PutBucketInventory.

Inventory lists

After you configure an inventory for a bucket, OSS generates inventory lists at the specified time interval. The following structure shows the directories in which generated inventory lists are stored.

dest_bucket
 └──destination-prefix/
     └──src_bucket/
         └──inventory_id/
             ├──YYYY-MM-DDTHH-MMZ/
             │   ├──manifest.json
             │   └──manifest.checksum
             └──data/
                 └──745a29e3-bfaa-490d-9109-47086afcc8f2.csv.gz

Directory

Description

destination-prefix/

This directory is generated based on the prefix specified when you configure an inventory. If no prefix is specified for inventory lists, this directory is omitted.

src_bucket/

This directory is generated based on the name of the bucket for which inventory lists are generated.

inventory_id/

This directory is generated based on the name of the inventory.

YYYY-MM-DDTHH-MMZ/

This directory indicates the start time when the bucket is scanned. The name of this directory is a timestamp in UTC. Example: 2020-05-17T16-00Z. The manifest.json and manifest.checksum objects are stored in this directory.

data/

Inventory lists that include the list of objects in the source bucket and the metadata of exported objects in the source bucket are stored in this directory. Inventory lists are CSV objects that are compressed by using Gzip.

Important
  • If a large number of objects are stored in the source bucket, OSS automatically splits the inventory lists into multiple CSV objects for downloading and processing. The names of these CSV objects are generated in the following format in sequence: uuid.csv.gz, uuid-1.csv.gz, and uuid-2.csv.gz. You can obtain the list of the CSV objects from the manifest.json object. Then, you can decompress the objects based on the preceding sequence to read the inventory lists.

  • The information about a single object is not split into multiple inventory lists.

After you configure an inventory for a bucket, the following objects are generated based on the inventory:

manifest objects

manifest objects include the manifest.json and manifest.checksum objects.

  • manifest.json: stores the metadata of inventory lists and related information.

    {
        "creationTimestamp": "1642994594",
        "destinationBucket": "destbucket",
        "fileFormat": "CSV",
        "fileSchema": "Bucket, Key, VersionId, IsLatest, IsDeleteMarker, Size, StorageClass, LastModifiedDate, ETag, IsMultipartUploaded, EncryptionStatus, ObjectAcl, TaggingCount, ObjectType, Crc64",
        "files": [{
                "MD5checksum": "F77449179760C3B13F1E76110F07****",
                "key": "destbucket/inventory0124/data/a1574226-b5e5-40ee-91df-356845777c04.csv.gz",
                "size": 2046}],
        "sourceBucket": "srcbucket",
        "version": "2019-09-01"}

    The following table describes the fields in the manifest.json object.

    Field

    Description

    creationTimestamp

    The start time when the source bucket is scanned. The value of this field is a UNIX timestamp.

    destinationBucket

    The bucket in which the inventory lists are stored.

    fileFormat

    The format of the inventory lists.

    fileSchema

    The fields in each inventory list. The fields are divided into fixed fields and optional fields. The sequence of the fixed fields is fixed. The sequence of the optional fields is determined by your selection sequence when you configure an inventory. We recommend that you parse the data columns in csv.gz based on the sequence of the fields in fileSchema. This prevents a mismatch between columns and attributes.

    • If you select the current object version when you configure an inventory, the fixed fields Bucket, Key in fileSchema are listed first, followed by the optional fields in fileSchema.

    • If you select all object versions when you configure an inventory, the fixed fields Bucket, Key, VersionId, IsLatest, IsDeleteMarker in fileSchema are listed first, followed by the optional fields in fileSchema.

    files

    The name, size, and MD5 hash of each inventory list. The name of an inventory list contains the full path.

    sourceBucket

    The source bucket for which the inventory is configured.

    version

    The version of the inventory list.

  • manifest.checksum: stores the MD5 hash of the manifest.json object. Example: 8420A430CBD6B659A1C0DFC1C11A****.

Inventory lists

Inventory lists contain the exported object information and are stored in the data/ directory. The following figure provides an example of an inventory list.

DCDD87B7-7499-4D22-B684-8B25B6F1C232.png

The sequence of the fields in the inventory list is determined by the sequence when you configure the inventory. The following table describes the fields in the preceding figure from left to right.

Field

Description

Bucket

The name of the source bucket for which the inventory is configured.

Key

The name of the object in the bucket.

The object name is URL-encoded. You must decode the object name before you can view the name.

VersionId

The version ID of the object.

This field exists only when versioning is enabled for the bucket and the inventory specifies that all versions of the object are included in the inventory list.

IsLatest

Specifies whether the version is the latest version. If the version is the latest version, this field is set to True. Otherwise, this field is set to False.

This field exists only when versioning is enabled for the bucket and the inventory specifies that all versions of data are exported.

IsDeleteMarker

Specifies whether the version is a delete marker. If the version is a delete marker, this field is set to True. Otherwise, this field is set to False.

This field exists only when versioning is enabled for the bucket and the inventory specifies that all versions of the object are included in the inventory list.

Size

The size of the object.

StorageClass

The storage class of the object.

LastModifiedDate

The time when the object was last modified in Greenwich Mean Time (GMT), which is 8 hours earlier than the UTC+8 time zone that is used in China.

ETag

The ETag of the object.

An ETag is generated when an object is created. The ETag is used to identify the content of the object.

  • If you create an object by calling the PutObject operation, the ETag of the object is the MD5 hash of the object content.

  • If you create an object by using other methods, the ETag of the object is not the MD5 hash of the object content but a unique value calculated based on the object.

IsMultipartUploaded

Specifies whether the object is created by using multipart upload. If you create the object by using multipart upload, the value of this field is True. Otherwise, the value is False.

EncryptionStatus

Specifies whether the object is encrypted. If the object is encrypted, the value of this field is True. Otherwise, the value is False.

ObjectAcl

The access control list (ACL) of the object. For more information, see Object ACLs.

TaggingCount

The number of tags of the object.

ObjectType

The type of the object. For more information, see Object ACLs.

Crc64

The CRC-64 of the object.

Usage notes

Suggestions on exporting inventory lists

You can export inventory lists on a daily or weekly basis.

Number of objects in a bucket

Export suggestions

< 10 billion

Export bucket inventory lists on a daily or weekly basis based on your business requirements.

10 billion to 50 billion

Export bucket inventory lists on a weekly basis.

≥ 50 billion

  • Export bucket inventory lists by prefix in several batches.

  • Submit a ticket to increase the number of inventory lists that can be exported.

Traffic and bandwidth

To increase the speed at which inventory lists are exported, bucket-level and user-level bandwidth may be occupied when the inventory lists are exported to the inventory storage bucket. If the bucket for which you want to configure the inventory is frequently accessed and the available bandwidth of the bucket is limited, we recommend that you create a bucket to store the inventory lists.

Exceptions

  • If no objects are stored in the bucket for which the inventory is configured or no objects match the specified prefix in the inventory, no inventory lists are generated.

  • When you export the inventory lists, the exported inventory lists may not contain all objects in the source bucket due to operations such as creation, deletion, or overwriting. If the time when an object was last modified is earlier than the time specified by the createTimeStamp field in the manifest.json object, inventory lists contain information about the object. Otherwise, the inventory lists may not contain information about the object. We recommend that you check the object attributes by calling the HeadObject operation before you export information about an object. For more information, see HeadObject.

Deletion of inventory lists

OSS continuously generates inventory lists based on the frequency specified by an inventory until the inventory is deleted. To prevent OSS from generating unnecessary inventory lists, you can delete inventories that you no longer need at the earliest opportunity. You can also delete exported historical inventory lists that you no longer need.

FAQ