Use rocketmq-spring-boot-starter to Configure, Send, and Consume RocketMQ Messages

By Liaotian

This article will briefly introduce the design and implementation of rocketmq-spring-boot. Readers can learn the development details of how to integrate the RocketMQ Client into the spring-boot-starter framework. This article will use a simple example to manifest how to use the spring-boot-starter toolkit to configure, send, and consume RocketMQ messages step by step.

A series on how to use RocketMQ in the Spring ecosystem: Lift Off with RocketMQ

In this article, you will learn about:

An Introduction to the Spring Messaging Framework
Implementation of rocketmq-spring-boot
Examples

Preface

In the late 1990s, Java EE (Enterprise Edition) emerged. The use of Enterprise Java Beans required complex descriptor configuration and rigid and complex code implementation, which increased the learning curve and development costs for developers. Therefore, the Spring technologies based on simple XML configuration and Plain Old Java Objects (POJO) were created. Technologies, such as Dependency Injection, Inversion of Control, and Aspect-Oriented Programming (AOP), solved the shortcomings of traditional Java EE in a more agile way.

As Spring continues to evolve, Annotation-based configurations gradually replace XML file configurations. Spring Boot 1.0.0 (officially released on April 1, 2014) can develop, test, run, and deploy Spring applications quickly based on the idea of convention over configuration. Combining with various initiators, such as spring-boot-web-starter, allows the application to run in command line mode directly without being deployed in a separate container. This easy, direct, and fast development process simplifies application deployment and allows developers to use specific configurations. Therefore, it is becoming increasingly popular among developers.

Apache RocketMQ is a well-known distributed middleware for message and stream processing. It consists of a Broker server and clients.

One client is the message Producer, which sends messages to the Broker server. The other is the message Consumer. Multiple Consumers can form a Consumer group to subscribe to, pull, and consume messages stored on the Broker.

The Apache RocketMQ community launched the spring-boot-starter implementation to use the rapid development feature of Spring Boot and to allow users to use the RocketMQ message client more flexibly. With the release of the distributed transaction message function in RocketMQ 4.3.0, the spring-boot code has been updated to support distributed transaction status checking and transaction message sending using annotations.

This article will give a brief introduction to the current design and implementation. Readers can learn the development details of integrating the RocketMQ client into the spring-boot-starter framework. This article will use a simple example to describe how to use the spring-boot-starter toolkit to configure, send, and consume RocketMQ messages step by step.

Message Framework in Spring

The two main frameworks on messages in Spring are Spring Messaging and Spring Cloud Stream. Both of them can be integrated with Spring Boot and provide some implementations for reference. Like all implementation frameworks, the messaging framework is designed to implement lightweight message-driven microservices, which can simplify developers' use of message-oriented middleware. This allows system developers to focus more on the processing of core business logic.

1. Spring Messaging

Spring Messaging is a module added in Spring Framework 4 and and adds extensible support for the integration of Spring and the message system. It implements a complete set of infrastructure from simple JMS interface usage based on JmsTemplate to asynchronous receiving of messages. Spring AMQP provides a similar set of functions required by the protocol. After being integrated with Spring Boot, it has the ability of automatic configuration and can integrate with the corresponding messaging system during testing and runtime.

For the client, Spring Messaging provides a set of abstract APIs or agreed standards to regulate the modes of message senders and receivers. Different message middleware providers can provide their own Spring implementations under this mode. A Java Bean in the form of XXXTemplate needs to be implemented on the message sender. Thus, it provides multiple different message sending methods based on the automatic configuration options of Spring Boot. The message consumer is an XXXMessageListener interface that provides callback methods to listen to and consume messages. It is implemented usually using an annotation to declare a message-driven POJO. This interface can also use Spring Boot automation options and some custom attributes.

By referring to existing implementations of Spring Messaging, the spring-boot-starter of RocketMQ follows the relevant design pattern and provides the corresponding APIs (such as order, asynchronous message, and transaction half message) based on the functional features of RocketMQ.

2. Spring Cloud Stream

Spring Cloud Stream combines the annotations and features of Spring Integration. Its application model is shown below:

This figure is quoted from Spring Cloud Stream.

The Spring Cloud Stream framework provides an independent application kernel that communicates with the outside world through @Input and @Output channels. The Source sends messages through input channels, while the Sink receives messages by listening to output channels. These channels connect to external proxies through a dedicated Binder. Developers only need to program their code based on the fixed interfaces and annotations provided by the application kernel without caring about the message-oriented middleware bound to the specific Binder at runtime. At runtime, Spring Cloud Stream can detect and use the Binder found under classpath automatically.

This allows developers to use different types of middleware in the same code easily. They simply need to put middleware in different Binders during building. In more complex scenarios, developers can package multiple Binders in an application and choose a Binder by itself. Different Binders can be used for different channels at runtime.

Binder abstraction allows Spring Cloud Stream applications to connect to the middleware flexibly. In addition, Spring Cloud Stream uses the flexible configuration capability of Spring Boot. Such configuration can be provided through externally configured properties and any object supported by Spring Boot, including application startup parameters, environment variables, and application.yml or application.properties files. The deployment staff can dynamically select the destination (such as Kafka topic or RabbitMQ exchange) for channel connection at runtime.

Binder SPI mode allows message middleware products to use extensible APIs to write corresponding Binders and integrate them into the Spring Cloud Steam environment. Currently, RocketMQ does not provide a Binder, but we plan to improve this function soon. We invite peers with relevant experience in the community to contribute PR or suggestions.

spring-boot-starter Implementation

We’ve already known that the starter built by spring boot starter is very convenient for users. Users only need to introduce the dependency definition of the starter in pom.xml. The corresponding compiling, run, and deployment functions will be introduced automatically. Therefore, common open-source components will provide an encapsulated spring-boot-starter for Spring users, which is very convenient for developers to integrate and use. Here, we introduce the implementation process of the RocketMQ (the client) starter in detail.

1. Steps for spring-boot-starter Implementation

The spring-boot starter implementation must include the following parts:

1) Definition in pom.xml

Define the starter component information to be generated:

<groupId>org.apache.rocketmq</groupId>
<artifactId>spring-boot-starter-rocketmq</artifactId>
<version>1.0.0-SNAPSHOT</version>

Define the dependency packages:

Two dependency packages are included: the Spring dependency package and RocketMQ dependency package.

2) Configure File Class

Define an application property configuration file class, RocketMQProperties. This Bean defines a set of default property values. When using the final starter, the user can modify the value according to the properties defined by the class. Of course, the configuration of this class is not modified, but the corresponding configuration file in the spring-boot application is src/main/resources/application.properties.

3) Define the Automatically Loaded Class

Define the automatically loaded class in src/resources/META-INF/spring.factories file. Its purpose is to enable spring boot to automatically initialize the related Bean, Component, or Service based on the automatic configuration class specified in this article. Its content is listed below:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.apache.rocketmq.spring.starter.RocketMQAutoConfiguration

In the implementation of the RocketMQAutoConfiguration class, the Bean objects open to users include:

RocketMQProperties: A processing class that loads the application property configuration file
RocketMQTemplate: A template class for the producer to send messages
ListenerContainerConfiguration container Bean: An interface class responsible for discovering and registering consumers for consumption. This class requires the RocketMQListener generic interface to be implemented. It also needs to be marked with the @RocketMQMessageListener annotation.

4) Perform Relevant RocketMQ Encapsulation

The spring-boot-starter will be encapsulated separately on the producer and the consumer. In the current implementation, a compatible method is provided for the Spring Messaging interface.

2. Message Producer Implementation

1) Common Producer

The code of the producer is encapsulated in RocketMQTemplate POJO. The following figure shows the call diagram of the relevant code of the producer:

To be compatible with Spring Messaging's sending template, the AbstractMessageSendingTemplate abstract class is integrated into RocketMQTemplate to support the relevant message conversion and sending methods. These methods will eventually connect to the doSend() method and some unique methods of RocketMQ, such as asynchronous, one-way, and sequential methods through proxies. By doing so, they are directly added to RocketMQTemplate. These methods will directly call the Producer API of RocketMQ to send messages.

2) Transaction Message Producer

The processing of transaction messages is partially extended on the message producer, as shown in the preceding call diagram.

In the RocketMQTemplate, a sendMessageInTransaction() method is added to send transaction messages. This method will eventually be called by the TransactionProducer of RocketMQ through a proxy. Its associated TransactionListener implementation class is registered on this Producer so that the method implementation in TransactionListener can be called after the message is sent.

3. Consumer Implementation

After the Spring-Boot application on the consumer is started, it scans all classes that contain the @RocketMQMessageListener annotation. These classes need to integrate with the RocketMQListener interface and implement the onMessage() method. The Listener will be placed in the DefaultRocketMQListenerContainer objects in a one-to-one manner. The container encapsulates the RocketMQListener into a specific RocketMQ concurrent or sequential API based on the consumption method (concurrent or sequential.) Create a RocketMQ Consumer object in the container and start and listen to customized Topic messages. If there are any consumed messages, the onMessage() method of the Listener will be called back.

Examples

The previous section describes how to use spring-boot-starter to implement RocketMQ. This section provides a simple example of sending and consuming messages to illustrate how to use this rocketmq-spring-boot-starter.

1. Prepare RocketMQ

1) Start the NameServer and Broker

Users need to make sure the RocketMQ service is correctly downloaded and started to verify the Spring-Boot client of RocketMQ. Please refer to the RocketMQ quick start to implement this step. Users must ensure that the NameServer and Broker have started correctly.

2) Create the Topics Required in the Instance

Run the following command in the directory where you run the startup command:

bash bin/mqadmin updateTopic -c DefaultCluster -t string-topic

2. Compile rocketmq-spring-boot-starter

The current spring-boot-starter dependencies are not submitted to the Maven central repository. You need to download the Git source code, run the mvn clean install command, and install it in a local repository:

git clone https://github.com/apache/rocketmq-externals.git
cd rocketmq-spring-boot-starter
mvn clean install

3. Write the Client Code

Users must add the following dependency to the maven configuration file pom.xml on the message producer and consumer to use it:

The value of the property spring-boot-starter-rocketmq-version is 1.0.0-SNAPSHOT, which is the same as the version of the local repository in the previous step.

1) Message Producer Code

The producer configuration file is application.properties:

Java code for the producer:

2) Message Consumer Code

The consumer configuration file is application.properties:

Java code for the consumer:

Here we’ve only introduced how to use spring-boot to write basic codes to send and receive messages. Please refer to the description documents and detailed code on GitHub to learn more about calling methods, such as asynchronous sending, object message body, specified tag, and specified transaction messages. We will also introduce these advanced functions in subsequent articles.

About the Author

Liaotian is an Alibaba Technical Expert and an Apache RocketMQ kernel fan. Liaotian has a deep understanding of Microservice, Messaging, and Storage with years of distributed system R&D experience. Currently, he focuses on RocketMQ kernel optimization and messaging ecosystem construction.

Community