All Products
Search
Document Center

Simple Message Queue (formerly MNS):OSS

Last Updated:Nov 22, 2024

Simple Message Queue (formerly MNS) allows you to create an event notification rule to push Object Storage Service (OSS) event notifications to a specific receiver. This topic describes the background information, scenarios, event types, and message format of OSS event notifications. This topic also describes how to create an event notification rule to push OSS event notifications.

Background information

After you create an event notification rule, when events are triggered on specific OSS resources, SMQ pushes event notifications to a specific receiver, as shown in the following figure.

image

When you send a request to call an operation of OSS, OSS adds a header to the response if an event notification rule is matched. This header indicates whether an event notification is generated. This does not affect the response to the original request. If no event notification rule is matched, no header is added. If an event notification rule is matched, a header is added regardless of whether an event notification is generated. Format of the header:

x-oss-process-status: 'ewogICAgImNvZGUiOiAiU3VjY2VzcyIsCiAgICAibWVzc2FnZSI6ICJOb3RpZmljYXRpb25TdWNjZWVkIiwKICAgICJ0eXBlIjogIkV2ZW50Tm90aWZpY2F****'
  • Key: x-oss-process-status.

  • Value: The value is encoded in Base64. Sample value after decoding:

    {
        "code":"Success", // Indicates whether an event notification is generated. If an event notification is generated, a value of Success is returned. Otherwise, a value of Fail is returned. A value of Success indicates that OSS sends a message to a MSQ topic but does not indicate that MSQ pushes the message from the topic to the endpoint of a receiver. 
        "message":"NotificationSucceed", // The description of the operation. If no event notification is generated, a failure cause is displayed. 
        "type":"EventNotification", // The type of the operation. The operation type of event notifications is EventNotification. 
        "version":"1.0"
    }

For more information about how to push notifications, see Queue.

Scenarios

  • If you authorize other users to perform operations on your OSS buckets and want to be notified when these operations occur, you can create an event notification rule.

  • You can analyze data based on event notifications, view the number of times for which an operation occurs within a period of time, and analyze the recent development trend of your business. You can also import event notifications into other services for analysis, such as E-MapReduce (EMR).

  • You can monitor the metrics such as QPS of operations that are performed on your OSS resources based on event notifications.

Events

Important

The ObjectCreatedGroup, ObjectDownloadedGroup, and ObjectRemovedGroup events are supported only in the following regions: China (Hong Kong), US (Silicon Valley), US (Virginia), Germany (Frankfurt), Singapore, and UK (London).

Event type

Description

Event name in a notification

PutObject

An object is created or overwritten by using simple upload.

ObjectCreated:PutObject

PostObject

An object is created or overwritten by using form upload.

ObjectCreated:PostObject

CopyObject

An object is created or overwritten by copying an object.

ObjectCreated:CopyObject

InitiateMultipartUpload

A multipart upload task is initiated.

ObjectCreated:InitiateMultipartUpload

UploadPart

An object is created or overwritten by using multipart upload.

ObjectCreated:UploadPart

UploadPartCopy

An object is created or overwritten by using multipart copy.

ObjectCreated:UploadPartCopy

CompleteMultipartUpload

A multipart upload task is complete.

ObjectCreated:CompleteMultipartUpload

AbortMultipartUpload

A multipart upload task is canceled and uploaded parts are deleted.

ObjectRemoved:AbortMultipartUpload

AppendObject

An object is created or appended by using append upload.

ObjectCreated:AppendObject

GetObject

An object is obtained by using simple download.

ObjectDownloaded:GetObject

DeleteObject

An object is deleted.

ObjectRemoved:DeleteObject

DeleteObjects

Multiple objects are deleted.

ObjectRemoved:DeleteObjects

ObjectReplication:ObjectCreated

An object is created in a data replication task.

ObjectReplication:ObjectCreated

ObjectReplication:ObjectRemoved

An object is deleted in a data replication task.

ObjectReplication:ObjectRemoved

ObjectReplication:ObjectModified

An object is overwritten in a data replication task.

ObjectReplication:ObjectModified

ObjectCreatedGroup

An object creation or overwrite operation is performed.

The event name in a notification can be any event that starts with ObjectCreated:, depending on the upload method that you select. For example, when an object is uploaded by using an OSS SDK by performing simple upload, the event name in a notification is ObjectCreated:PutObject. When an object is uploaded in the OSS console, the event name in a notification is ObjectCreated:PostObject.

ObjectDownloadedGroup

An object download operation is performed.

The event name in a notification can be any event that starts with ObjectDownloaded:, depending on the download method that you select. For example, when an object is downloaded by performing simple upload, the event name in a notification is ObjectDownloaded:GetObject.

ObjectRemovedGroup

An object deletion operation is performed.

The event name in a notification can be any event that starts with ObjectRemoved:, depending on the deletion method that you select. For example, when an object is deleted, the event name in a notification is ObjectRemoved:DeleteObject. When multiple objects are deleted at a time, the event name in a notification is ObjectRemoved:DeleteObjects.

FinishRestore

An object restoration operation is complete. The event type supports only Cold Archive and Deep Cold Archive objects.

ObjectRestore:FinishRestore

Message format

The message content of an OSS event notification is encoded in Base64. The decoded content is in the JSON format. The following sample code provides an example of the decoded message content.

{
    "events":[
        {
            "eventName":"ObjectDownloaded:GetObject",
            "eventSource":"acs:oss",
            "eventTime":"2016-07-01T11:17:30.000Z",
            "eventVersion":"1.0",
            "oss":{
                "bucket":{
                    "arn":"acs:oss:cn-shenzhen:123456789098****:event-notification-test-shenzhen",
                    "name":"event-notification-test-shenzhen",
                    "ownerIdentity":"123456789098****"
                },
                "object":{
                    "deltaSize":0,
                    "eTag":"0CC175B9C0F1B6A831C399E26977****",
                    "key":"test",
                    "readFrom":0,
                    "readTo":1,
                    "size":1
                },
                "ossSchemaVersion":"1.0",
                "ruleId":"GetObjectRule"
            },
            "region":"cn-shenzhen",
            "requestParameters":{
                "sourceIPAddress":"140.205.XX.XX"
            },
            "responseElements":{
                "requestId":"5776514AF09A9E654242****"
            },
            "userIdentity":{
                "principalId":"123456789098****"
            },
            "xVars":{
                "x:callback-var1":"value1",
                "x:vallback-var2":"value2"
            }
        }
    ]
}

The following table describes the parameters in the message content.

Parameter

Description

eventName

The event type.

eventSource

The event source.

eventTime

The time when the event occurred.

eventVersion

The version of the event protocol.

oss

The content of the OSS event.

bucket

The details of the OSS bucket.

arn

The Alibaba Cloud Resource Name (ARN) of the bucket.

name

The bucket name.

ownerIdentity

The ID of the user who created the bucket.

object

The details of the object.

deltaSize

The size change of the object.

eTag

The content of the object. If an object is created by using the PutObject request, the entity tag (ETag) of the object is the MD5 hash of the object content. If an object is created by using other methods, the ETag is a unique value generated based on a specific algorithm. The ETag of an object can be used to check whether the object content is modified.

key

The object name.

readFrom

The position where the object read starts.

readTo

The position where the object read ends.

size

The size of the object.

ossSchemaVersion

The version of the OSS schema.

ruleId

The ID of the rule that matches the event.

region

The region in which the bucket resides.

requestParameters

The values of the request parameters.

sourceIPAddress

The source IP address of the request.

responseElements

The values of the response parameters.

requestId

The request ID.

userIdentity

The user information.

principalId

The unique ID (UID) of the requester.

xVars

The custom parameters in the callback of OSS.

Usage notes

  • It takes approximately 10 minutes for a new event notification rule to take effect.

  • The system automatically creates a topic for an event notification rule, and you may be charged for the created topic. For more information, see Billing.

  • You can create or delete event notification rules. However, you cannot modify an event notification rule.

Step 1: Create a queue

  1. Log on to the SMQ console.

  2. In the left-side navigation pane, choose Queue Model > Queues.

  3. In the top navigation bar, select a region.

  4. On the Queues page, click Create Queue.

  5. In the Create Queue panel, configure the following parameters and click OK:

    • Name: the name of the queue.

    • Maximum Message Length: the maximum length of the message that is sent to the queue.

    • Long Polling Period: the maximum duration for which long polling requests are held after the ReceiveMessage operation is called.

    • Visibility Timeout Period: the duration for which a message stays in the Inactive state after the message is received from the queue.

    • Message Retention Period: the maximum duration for which a message exists in the queue. After the specified retention period, the message is deleted regardless of whether the message is received.

    • Scheduled Period: the period after which all messages sent to the queue are consumed.

    • Enable Logging Feature: specifies whether to enable the logging feature.

    After the queue is created, it is displayed on the Queues page.

Step 2: Create a bucket

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets.

  3. On the Buckets page, click Create Bucket.

  4. In the Create Bucket panel, configure the following parameters and click Create.

    • Bucket Name: the name of the bucket. In this example, event-notification-test is used.

    • Region: the region in which the bucket resides. The region must be the same as the region in which the queue resides.

    • Block Public Access: specifies whether to block public access. In this example, this feature is disabled.

    • ACL: specifies the permissions on the bucket. In this example, Public Read/Write is selected.

      Note

      Use the default values of other parameters.

Step 3: Create an event notification rule

After an event notification rule is created, SMQ automatically creates a topic for the rule and creates a subscription to the topic with the specified endpoint. You can view and manage topics and subscriptions in the SMQ console.

  1. Log on to the SMQ console.

  2. In the left-side navigation pane, click Event Notifications.

  3. In the top navigation bar, select a region.

  4. On the OSS tab of the Event Notifications page, click Create Rule.

  5. In the Create Rule - OSS panel, configure the following parameters and click OK.

    • Name: Enter a name for the rule.

    • Event Type: Select one or more event types.

    • Match Mode: Configure the match mode for the rule.

      Note

      Match modes include Full Name, Prefix, Suffix, and Prefix & Suffix. Two match modes cannot have intersections. OSS resources include buckets and objects. Their names are separated with forward slashes (/). Example: movie/hello.avi.

      • Full Name: movie/hello.avi.

      • Prefix:

        • movie/hello indicates all objects with a prefix of hello in the movie bucket.

        • movie/ indicates all objects in the movie bucket.

        • mov indicates all objects in the buckets with a prefix of mov.

      • Suffix: avi indicates all objects with a suffix of .avi in all buckets.

      • Prefix & Suffix: A combination of the movie/ prefix and the .avi suffix indicates all objects with a suffix of .avi in the movie bucket.

    • Receiving Terminal: Select Queue from the drop-down list and enter the name of the queue that you created in Step 1.

Step 4: Upload an object

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets.

  3. On the Buckets page, click the name of the OSS bucket that you want to manage to go to the Object Management page.

  4. Click Upload Object above the object list. >

  5. On the page that appears, configure the parameters and click Upload Object. For more information about the parameters, see the Use the OSS console section of the "Simple upload" topic.

    The Task List panel appears. On the Upload Tasks tab, the state of the object is displayed as Uploaded.

Step 5: Receive messages

  1. Log on to the SMQ console.

  2. In the left-side navigation pane, choose Queue Model > Queues.

  3. In the top navigation bar, select a region.

  4. On the Queues page, find the queue that you want to manage and choose More > Send Messages in the Action column.

  5. In the Receive Message section of the Quick Experience page, click Receive Message.

    A list of messages appears in the Receive Message section.

  6. In the message list, find the message that you want to view and click Details in the Actions column. In the Message Details dialog box, view the message content.