Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog RocketMQ 5.0 VS 4.9.X: Comparison of Architecture Diagram

RocketMQ 5.0 VS 4.9.X: Comparison of Architecture Diagram

This article introduces the comparison of the architecture diagrams between RocketMQ version 5.0 and version 4.9.X.

By Wei Li, Apache RocketMQ Committer, RocketMQ Python Client Project Owner, Apache Doris Contributor, and TencentDB Development Engineer

Architecture of RocketMQ 4.9.X

1

The following table briefly describes the communication between each component in 4.9.X.

Component and data flow Description
Namesrv A stateless application that stores the topic routing information. Topic route: topic-queue-broker.
Broker A stateful application that handles data computation and storage. The data it computes includes producer requests, consumer requests, management requests, and broker system services, such as index building and message expiry. The data it stores includes messages and indexes.
Broker -> namesrv The broker periodically reports the broker information and the topic information in the current broker to the namesrv.
Producer An application that produces messages.
Producer <-> namesrv The producer obtains the topic routing information that includes the broker IP address from the namesrv.
Producer <-> broker The producer sends messages directly to the broker by using topic routing information. The producer periodically sends heartbeats to the broker to report the current producer instance information.
Consumer An application that consumes messages.
Consumer <-> namesrv The consumer obtains the topic routing information that includes the broker IP address from the namesrv.
Consumer <-> broker The consumer pulls messages from the specified broker for consumption by using topic routing information. The consumer periodically sends heartbeats to the broker to report the current consumer instance information.
  • The producer routes messages through the topic and send them directly to the broker.
  • The producer periodically sends heartbeats to the broker to report the current producer instance information.

Consumer consumes messages. Consumer <-> namesrv

  • The consumer obtains the topic routing information that includes the broker IP address from the namesrv.

Consumer <-> broker

  • The consumer pulls messages from the specified broker based on routing information.
  • The consumer periodically sends heartbeats to the broker to report the current consumer instance information.

Architecture of RocketMQ 5.0.0

2

The following table briefly describes the communication between each component in 5.0.0.

Component and data flow Description
Namesrv A stateless application that stores the topic routing information. Topic route: topic-queue-broker. In 5.0.0, the namesrv supports controller modules. If you set the enableControllerInNamesrv value to true, a controller instance gets embedded and started in the namesrv process.
Broker A stateful application that handles data computation and storage. The data it computes includes producer requests, consumer requests, management requests, and broker system services, such as index building and message expiry. The data it stores includes messages and indexes. In 5.0.0, the broker can perform primary, secondary, and learner switchover. If you set the asyncLearner value to true, the broker switches to the learner mode. It only synchronizes data and does not elect the master.
Broker -> namesrv The broker periodically reports the broker information and the topic information in the current broker to the namesrv.
Remoting producer An application that produces messages.
Remoting producer <-> namesrv The producer obtains the topic routing information that includes the broker IP address from the namesrv.
Remoting producer <-> broker The producer sends messages directly to the broker by using topic routing information. The producer periodically sends heartbeats to the broker to report the current producer instance information.
Remoting consumer An application that consumes messages.
Remoting consumer <-> namesrv The consumer obtains the topic routing information that includes the broker IP address from the namesrv.
Remoting consumer <-> broker The consumer pulls messages from the specified broker for consumption by using topic routing information. The consumer periodically sends heartbeats to the broker to report the current consumer instance information.
New module in 5.0.0, controller Similar to the Kafka controller, the RocketMQ controller is responsible for electing the broker master and notifying all brokers.
New module in 5.0.0, broker <-> controller The broker periodically reports the broker information to the Controller. The broker synchronizes the replica status. After the controller elects a new broker master, it notifies all brokers.
New Module in 5.0.0, proxy A stateless application. The new client accesses the proxy through the gRPC interface to send and subscribe to messages. The Remoting Protocol is supported in the community. The proxy supports brokers. If you set the proxyMode value to LOCAL, a broker instance gets started in the proxy process.
New module in 5.0.0, proxy <-> broker The proxy communicates with the broker by using the Remoting Protocol. You can use the proxy as a remoting client.
New Module in 5.0.0, new client The new client supports production, consumption, and management features. It also supports the gRPC Protocol.
New module in 5.0.0, new client <-> proxy The new client accesses the proxy to send, subscribe to, and manage messages.
  • The producer sends messages directly to the broker through topic routing information.
  • The producer periodically sends heartbeats to the broker to report the current producer instance information.

Remoting consumers consume messages. Remoting consumers <-> namesrv

  • The consumer obtains topic routing information from Namesrv, including Broker IP Remoting Consumer <-> Broker.
  • The consumer pulls messages from the specified broker based on topic routing information.
  • The consumer periodically sends heartbeats to the broker to report the current consumer instance information.

New Module in 5.0.0

Controller in RocketMQ is similar to Kafka Controller, responsible for electing and notifying the Broker Master.

New Module in 5.0.0

Broker <-> Controller

  1. The broker periodically reports broker information to the controller.
  2. The broker synchronizes the replica status.
  3. After the controller elects a new broker master, it notifies all brokers.

New Module in 5.0.0

Proxy is a stateless application. The new clients access the Proxy through the gRPC interface to send and receive messages.

Proxy supports embedding Broker.

If proxyMode=LOCAL is set, a Broker instance will be started within the Proxy process.

New Module in 5.0.0

Proxy <-> BrokerProxy

Communicates with the Broker using the Remoting protocol, where Proxy acts as a Remoting client.

New Module in 5.0.0

New Client

A new client that supports production, consumption, and management functions. Currently supports the gRPC Protocol.

New Module in 5.0.0

New Client <-> Proxy

The new client accesses the Proxy for message sending, receiving, and management.

middleware Architecture RocketMQ Computing Developers Message Queue cloud native Apache RocketMQ
0 1 0
Share on

Read previous post:

Introduction to Nacos 2.3.0-Beta Version

Read next post:

How to Configure TLS-encrypted Transmission in RocketMQ 5.0?

Alibaba Cloud Native Community

508 posts | 48 followers

You may also like

Comments

Alibaba Cloud Native Community

508 posts | 48 followers

Related Products

More Posts by Alibaba Cloud Native Community

See All