Create custom rules to implement OSS event notifications - Simple Message Queue (formerly MNS)

Simple Message Queue (SMQ, formerly MNS) pushes notifications when specific operations occur on your Object Storage Service (OSS) resources. Set up an event notification rule to route these notifications to a queue for downstream processing.

How it works

After you create an event notification rule, SMQ monitors the specified OSS resources for matching events. When an event occurs, SMQ pushes a notification to the receiver you configured, such as a queue.

OSS adds an x-oss-process-status response header to requests that match an event notification rule. This header is present regardless of whether the notification was delivered successfully. This does not affect the response to the original request. Requests that do not match any rule do not include this header.

The header value is Base64-encoded. After decoding, the JSON payload contains the following fields:

{
    "code": "Success",
    "message": "NotificationSucceed",
    "type": "EventNotification",
    "version": "1.0"
}

Field	Description
`code`	Whether the notification was generated. `Success` means OSS sent the message to an SMQ topic. This does not confirm that SMQ delivered the message to the endpoint. `Fail` indicates the notification was not generated.
`message`	Result description. On failure, this field contains the cause.
`type`	Operation type. Always `EventNotification` for event notifications.
`version`	Protocol version.

For more information about message delivery, see Queue.

Use cases

Track operations on shared buckets -- Get notified when authorized users upload, delete, or modify objects in your buckets.
Analyze access patterns -- Aggregate notifications to measure operation frequency, identify trends, or feed data into analytics services such as E-MapReduce (EMR).
Monitor request rates -- Track QPS for operations on your OSS resources.

Usage notes

A new event notification rule takes approximately 10 minutes to take effect.
SMQ automatically creates a topic for each event notification rule. The topic may incur charges. For details, see Billing overview.
Event notification rules can be created or deleted, but not modified. To update a rule, delete it and create a new one.

Supported event types

Events are grouped by category. Use the wildcard suffix :* to match all events in a category, including any future event types added to that category.

Object creation events

Event type	Trigger
ObjectCreated:PutObject	PutObject uploads an object.
ObjectCreated:PostObject	PostObject uploads an object via an HTML form.
ObjectCreated:CopyObject	CopyObject copies an object within or between buckets.
ObjectCreated:AppendObject	AppendObject appends data to an existing object.
ObjectCreated:InitiateMultipartUpload	InitiateMultipartUpload starts a multipart upload task.
ObjectCreated:UploadPart	UploadPart uploads a part in a multipart upload.
ObjectCreated:UploadPartCopy	UploadPartCopy copies data as a part in a multipart upload.
ObjectCreated:CompleteMultipartUpload	CompleteMultipartUpload completes a multipart upload.
ObjectCreated:PutSymlink	PutSymlink creates a symbolic link to a destination object.
ObjectCreated:*	Matches all object creation events, including future event types that start with `ObjectCreated:`.

Object download events

Event type	Trigger
ObjectDownloaded:GetObject	GetObject downloads an object.

Object modification events

Event type	Trigger
ObjectModified:UpdateObjectMeta	UpdateObjectMeta updates object metadata.
ObjectModified:ChangeStorageClass	A lifecycle rule changes the storage class of an object.
ObjectModified:*	Matches all object modification events, including future event types that start with `ObjectModified:`.

Object removal events

Event type	Trigger
ObjectRemoved:DeleteObject	DeleteObject deletes an object.
ObjectRemoved:DeleteObjects	DeleteMultipleObjects deletes multiple objects in a single request.
ObjectRemoved:AbortMultipartUpload	AbortMultipartUpload cancels a multipart upload and deletes the uploaded parts.
ObjectRemoved:*	Matches all object removal events, including future event types that start with `ObjectRemoved:`.

Data replication events

Event type	Trigger
ObjectReplication:ObjectCreated	An object is created during a data replication task.
ObjectReplication:ObjectRemoved	An object is deleted during a data replication task.
ObjectReplication:ObjectModified	An object is overwritten during a data replication task.
ObjectReplication:*	Matches all data replication events, including future event types that start with `ObjectReplication:`.

Object restore events

Event type	Trigger
ObjectRestore:FinishRestore	A Cold Archive or Deep Cold Archive object restoration completes. This event type applies only to Cold Archive and Deep Cold Archive objects.

Event notification message format

The message content is Base64-encoded. The decoded payload is a JSON object with the following structure:

{
    "events": [
        {
            "eventName": "ObjectDownloaded:GetObject",
            "eventSource": "acs:oss",
            "eventTime": "2016-07-01T11:17:30.000Z",
            "eventVersion": "1.0",
            "oss": {
                "bucket": {
                    "arn": "acs:oss:cn-shenzhen:123456789098****:event-notification-test-shenzhen",
                    "name": "event-notification-test-shenzhen",
                    "ownerIdentity": "123456789098****"
                },
                "object": {
                    "deltaSize": 0,
                    "eTag": "0CC175B9C0F1B6A831C399E26977****",
                    "key": "test",
                    "readFrom": 0,
                    "readTo": 1,
                    "size": 1
                },
                "ossSchemaVersion": "1.0",
                "ruleId": "GetObjectRule"
            },
            "region": "cn-shenzhen",
            "requestParameters": {
                "sourceIPAddress": "140.205.XX.XX"
            },
            "responseElements": {
                "requestId": "5776514AF09A9E654242****"
            },
            "userIdentity": {
                "principalId": "123456789098****"
            },
            "xVars": {
                "x:callback-var1": "value1",
                "x:vallback-var2": "value2"
            }
        }
    ]
}

Message fields

Field	Description
eventName	Event type that triggered the notification.
eventSource	The event source. The value is `acs:oss`.
eventTime	The time when the event occurred, in ISO 8601 format.
eventVersion	Event protocol version.
oss	OSS event details.
oss.bucket.arn	Alibaba Cloud Resource Name (ARN) of the bucket.
oss.bucket.name	Bucket name.
oss.bucket.ownerIdentity	The ID of the user who created the bucket.
oss.object.deltaSize	The size change of the object, in bytes.
oss.object.eTag	ETag of the object. For objects uploaded with PutObject, this is the MD5 hash of the content. For objects created through other methods, this is a unique value generated by an internal algorithm. Compare ETags to check whether object content has changed.
oss.object.key	Object name.
oss.object.readFrom	The starting byte position of the object read.
oss.object.readTo	The ending byte position of the object read.
oss.object.size	The size of the object, in bytes.
oss.ossSchemaVersion	OSS schema version.
oss.ruleId	ID of the event notification rule that matched this event.
region	Region where the bucket resides.
requestParameters.sourceIPAddress	The source IP address of the request.
responseElements.requestId	Request ID.
userIdentity.principalId	The unique ID (UID) of the requester.
xVars	Custom parameters defined in the OSS callback configuration.

Prerequisites

Before you begin, make sure that you have:

An Alibaba Cloud account with SMQ and OSS activated
(Optional) A queue to receive notifications. If you do not have one, create it in Step 1
(Optional) An OSS bucket in the same region as the queue. If you do not have one, create it in Step 2

Procedure

Step 1: (Optional) Create a queue

Skip this step if you already have a queue to receive event notifications. You can also create a subscription after the event notification rule is created.

Log on to the SMQ console.
In the left-side navigation pane, choose Queue Model > Queues.
In the top navigation bar, select a region.
On the Queues page, click Create Queue.

In the Create Queue panel, configure the following parameters and click OK: The new queue appears on the Queues page.

Parameter	Description
Name	Queue name.
Maximum Message Length	Maximum size of a single message sent to the queue.
Long Polling Period	The maximum duration for which long polling requests are held after the ReceiveMessage operation is called.
Visibility Timeout Period	The duration for which a message stays in the Inactive state after the message is received from the queue.
Message Retention Period	Maximum time a message is retained in the queue. After this period, the message is deleted whether or not it has been received.
Message Delay Period	The period after which all messages sent to the queue are consumed.
Enable Logging Feature	Whether to enable message operation logging.

Step 2: (Optional) Create a bucket

Skip this step if you already have a bucket in the same region as the queue.

Log on to the OSS console.
In the left-side navigation pane, click Buckets.
On the Buckets page, click Create Bucket.
In the Create Bucket panel, configure the following parameters and click Create:
Use the default values for all other parameters. The bucket is created with Block Public Access enabled and ACL set to Private by default. These settings are sufficient for event notification testing.
Parameter
Description
Bucket Name
Name of the bucket. Example: event-notification-test.
Region
Must be the same region as the queue.

Step 3: Create an event notification rule

When you create an event notification rule, SMQ automatically creates a topic and a subscription that routes messages from the topic to the specified receiver.

Log on to the SMQ console.
In the left-side navigation pane, click Event Notifications.
In the top navigation bar, select a region.
On the OSS tab of the Event Notifications page, click Create Rule.

In the Create Rule - OSS panel, configure the following parameters and click OK:

Parameter	Description
Name	A name for the rule.
Event Type	One or more event types to monitor. See Supported event types.
Match Mode	Filtering pattern for OSS resource names. See Match mode details.
Receiving Terminal	Select Queue and enter the name of the queue created in Step 1.

Match mode details

OSS resource names combine bucket and object names separated by a forward slash (/). For example: movie/hello.avi.

Four match modes are available. Two match modes cannot have intersections.

Match mode	Example	What it matches
Full Name	`movie/hello.avi`	Only the exact resource `movie/hello.avi`.
Prefix	`movie/hello`	All objects with a `hello` prefix in the `movie` bucket.
Prefix	`movie/`	All objects in the `movie` bucket.
Prefix	`mov`	All objects in buckets whose names start with `mov`.
Suffix	`avi`	All objects with an `.avi` suffix across all buckets.
Prefix & Suffix	Prefix: `movie/`, Suffix: `.avi`	All `.avi` objects in the `movie` bucket.

Step 4: Upload an object to verify the rule

Wait at least 10 minutes after creating the rule before uploading, because new rules take approximately 10 minutes to take effect.

Log on to the OSS console.
In the left-side navigation pane, click Buckets.
Click the target bucket name to open the Objects page.
Click Upload Object above the object list.
Configure the upload parameters and click Upload Object. For details, see the Simple upload section. The Task List panel shows the upload status as Uploaded.

Step 5: Receive the event notification

Log on to the SMQ console.
In the left-side navigation pane, choose Queue Model > Queues.
In the top navigation bar, select the same region as the queue.
On the Queues page, find the queue and click Send Messages in the Actions column.
In the Receive Message section of the Quick Experience page, click Receive Message. A list of received messages appears.
Find the message and click Details in the Actions column. The Message Details dialog box displays the event notification content.

Parameter	Description
Bucket Name	Name of the bucket. Example: `event-notification-test`.
Region	Must be the same region as the queue.