Since its development about a decade ago, Message Queue for Apache RocketMQ has been proven to be a stable solution for a wide variety of businesses. Now, it is widely implemented in Alibaba Group's internal processes and used by tens of thousands of Alibaba Cloud enterprise users. As a messaging solution with financial-grade reliability, Message Queue for Apache RocketMQ has always focused on building its asynchronous communication capacity in business integration. This article introduces the core capabilities and advantages of using Message Queue for Apache RocketMQ as a messaging solution for business integration. This article also describes the use scenarios, use cases, and best practices of the normal messages feature in Message Queue for Apache RocketMQ.
A typical business integration scenario is the e-commerce scenario. As large-scale sales promotions are often carried out in the e-commerce scenario, high requirements are placed on the throughput, end-to-end latency, and load shifting of the messaging system. Against this backdrop, Message Queue for Apache RocketMQ was developed and was first used in Alibaba Group's e-commerce systems.
This article aims to provide some insights into the following questions:
When designing core business architectures, we must design the business logic required by the business. An example of business integration is the e-commerce transaction scenario. In this scenario, the system may be split into multiple applications using the microservice architecture. When we integrate these applications using messages, we focus on the business logic that says the orders from the users are processed as expected.
In contrast, data integration is data-centric and focuses more on the analysis of the data generated by business integration. Data integration focus on the properties and the relationship between data instead of where data originates.
The coupling between the caller and the callee increases in business integration scenarios as the business logic of an enterprise expands in scope and increases in complexity, and the system typology becomes more complex. In many cases, the upstream application of a message may be the downstream application of another message, or a service may be both a message producer and a message consumer.
However, in data integration scenarios, we focus more on the heterogeneity and diversity of data instead of the system itself. First, we extract and cleanse data from various heterogeneous data sources to perform data integration analysis and then aggregate the data into structured data or reports for further analysis.
Business integration can be viewed as online logic or hard real-time logic. The caller and the callee must responsd in a timely manner to both synchronous and asynchronous invocations in business integration scenarios. For example, an order must be processed in milliseconds. Otherwise, the user experience is poor.
However, data integration is usually used in near real-time or even offline (non-real-time) scenarios. Users crawl data from batch processing, real-time streaming, or near real-time streaming scenarios for further analysis. The backend processing is invisible to users. This also reflects the focus difference between data integration and business integration.
The message queue paradigm is a major business integration mode used by enterprises. Message queues implement an asynchronous communication pattern and provide capabilities that feature low coupling, high reliability, and observability. What are the benefits of using messages in the business integration system? I will briefly describe them in the following section.
The preceding figure shows a typical upper-layer application system in which a message is sent from Application A to Application B. After the message is initialized or structured, the message is sent to the event channel (such as Message Queue for Apache RocketMQ and Message Queue for RabbitMQ). This channel is known as the messaging system. The message is stored in the event channel and then matches and pushes to the downstream application using the distribution components of the router. Meanwhile, the observability, O&M, and monitoring features are provided to ensure the stable running of the application system.
The desired capabilities in the messaging system of business integration can be many. Here are the four core capabilities:
As a typical messaging solution for business integration, Message Queue for Apache RocketMQ provides a full range of messaging features, a variety of clients, a complete observability system, and a stability assurance solution to meet the preceding demands of business integration.
This article describes one of the multiple types of messages in Message Queue for Apache RocketMQ — normal messages.
Normal messages are the simplest yet most important type of messages in Message Queue for Apache RocketMQ. As a basic message type, normal messages provide an asynchronous communication capability that features high throughput, high scalability, and low latency. Other advanced message types are produced by adding unique features to or specifying the usage of normal messages.
The following figure shows a typical topology of normal messages. As you can see, producers first send normal messages to the Message Queue for Apache RocketMQ broker for storage. Then, the broker matches and pushes these messages to consumers based on their subscriptions.
The lifecycle of normal messages starts with message initialization and ends with message deletion. Understanding the lifecycle of normal messages helps us locate and troubleshoot issues.
The status of a normal message can generally be divided into the following five types:
I briefly introduced the working mechanism of ordered messages in the previous section. What are their typical use scenarios? Among all types of messages in Message Queue for Apache RocketMQ, normal messages are the most widely and largely used. This type of message is primarily used in scenarios (such as service decoupling and calling), as well as batch data collection and transmission.
1) Microservice Calling and Decoupling
2) Real-Time Data Transmission
1) Scenario
A trading platform is an online system where people can exchange money for goods based on agreed terms and conditions. A trading process involves the interaction between multiple subsystems, such as payment, logistics, ordering, and operations. Normal messages are used for asynchronous decoupling to ensure the stability and efficiency of the trading process, especially during large-scale online sales promotions.
2) Pain Points
Complex Order State Machine: The lifecycle of orders is long, as they are forwarded between multiple downstream subsystems. Synchronous invocations can be time-consuming and lead to a bad user experience.
Huge Numbers of Orders: Huge numbers of orders need to be processed in a large-scale sales promotion scenario. The insufficient processing capability of subsystems may result in a system crash.
Persistence of Order Changes and Transactional Consistency for Downstream Invocations in Distributed Scenarios: The forwarding of order status requires that the status change of the database is consistent with downstream invocations, which is known as transactional consistency.
We learned about the use scenarios and cases for Message Queue for Apache RocketMQ in the preceding sections. Now, let's take a look at how to use the code.
The process for sending normal messages is very simple. However, we must take note of the following points:
Message Queue for Apache RocketMQ supports both the active and passive consumption modes.
If you use the passive consumption mode to consume messages, all you need to do is to register a consumption listener. Then, the listener processes the consumption logic and returns the consumption result. The result of the consumption failure is required if you want Message Queue for Apache RocketMQ to retry a message. If an exception is thrown, this is also considered a returned result. When such results are returned to the broker, the consumption is completed.
Compared with the passive consumption mode, the active consumption mode is more flexible. Consumers can actively obtain messages and concurrently consume the messages based on their business requirements. The consumption results are returned to the Message Queue for Apache RocketMQ broker after the consumers finish the consumption.
How China Everbright Bank Uses the RocketMQ-Based Distributed Messaging Platform
507 posts | 48 followers
FollowAlibaba Cloud Native Community - February 7, 2023
Alibaba Cloud Native Community - December 19, 2022
Alibaba Cloud Native Community - February 16, 2023
Alibaba Cloud Native - June 7, 2024
Alibaba Cloud Native Community - February 1, 2023
Alibaba Cloud Native Community - November 20, 2023
507 posts | 48 followers
Follow
ApsaraMQ for RocketMQ
ApsaraMQ for RocketMQ is a distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications.
Learn More
Message Queue for Apache Kafka
A fully-managed Apache Kafka service to help you quickly build data pipelines for your big data analytics.
Learn More
AliwareMQ for IoT
A message service designed for IoT and mobile Internet (MI).
Learn More
Message Queue for RabbitMQ
A distributed, fully managed, and professional messaging service that features high throughput, low latency, and high scalability.
Learn MoreMore Posts by Alibaba Cloud Native Community
