Manage the visibility timeout of a message

Background information

Simple Message Queue (formerly MNS) provides the VisibilityTimeout parameter for each message. After a worker process receives a message, SMQ starts to time for the visibility timeout. If a worker process does not finish processing a message within the specified timeout period, the message may be received and processed by other worker processes.

If you specify the VisibilityTimeout parameter for a message and a worker process fails to process a message, the message is not deleted and can be processed by other worker processes.

You can set a large value for the VisibilityTimeout parameter to ensure that a worker process has enough time to process messages.

Scenario

Assume that the value of the VisibilityTimeout parameter for a queue is set to 6 hours.

A worker process receives a message M1. After the worker process processes the message, the process fails or the host is restarted.

In this case, the message M1 cannot be received or processed by another worker process until six hours later. You can write code to shorten the time, but this will make the process more complicated.

Requirements

In specific scenarios, you need to process messages immediately after they are received. Assume that you have the following requirements:

• The value of the VisibilityTimeout parameter is small, for example, 5 minutes. If a worker process fails, messages that have not been processed by the worker process can be received and processed by other worker processes within five minutes.

• When a worker process processes a message, the connection must be maintained. This is because the process may take more than five minutes.

Solution

SMQ provides the C# sample code to resolve the issue described in the preceding scenario. Download the sample code. When a worker process processes a message, SMQ regularly checks whether to call the ChangeVisibility operation. After the worker process processes the message, the deleteMessage operation is called to delete the message.

For technical support about the demo usage,submit a ticket.

When you use the sample code, take note of the following information:

• Before you run the code, you must configure the AccessKey ID, AccessKey secret, and endpoint.

• Variables

  • MessageMinimalLife: the minimum lifetime that is required when a message is registered. For example, if a message will time out 0.1 seconds after it is registered, SMQ does not have sufficient time to call the ChangeVisibility operation to extend the lifetime of the message. Therefore, MessageMinimalLife ensures that a message does not time out before SMQ can call the ChangeVisibility operation to extend the lifetime of the message. You can specify the MessageMinimalLife parameter based on your business needs.

  • TimerInterval: the interval that the internal timer of the manager uses for timing. You must make sure that the timer is started before the specified lifetime of a message ends. You can set a small value for the TimerInterval parameter so that messages are checked at short intervals.

  • QueueMessageVisibilityTimeout: the default timeout period of a message in a queue. When you use the sample code to call the ChangeVisibility operation, the value of the VisibilityTimeout parameter is the same as the one specified for the QueueMessageVisibilityTimeout parameter. Therefore, the value of the QueueMessageVisibilityTimeout parameter must be greater than the sum of the values of the TimerInterval and MessageMinmalLife parameters. Otherwise, the message may time out.

  • MessageTimeout: the timeout period of a message in the manager. For example, if a worker process fails, the manager no longer changes the visibility of the message. Then, the message times out.

• Process

  • After a message is received, a worker process registers and processes the message, and then calls the Manager.deleteMessage operation to delete the message.

  • After a message is registered, the manager calls the ThreadPool.ChangeVisibilityTask operation to check whether the ChangeVisibility operation is required. Then, the manager adds the message to the internal message list.

  • The timer in the manager regularly calls the Parallel.ChangeVisibilityTask operation to check all messages in the message list.

  • The following figure describes the process that is used to manage the visibility timeout of messages by calling the Manager.ChangeMessageVisibility (ChangeVisibilityTask) operation.