ApsaraMQ for RocketMQ:Migrate a self-managed Apache RocketMQ cluster to ApsaraMQ for RocketMQ

Source cluster requirements

The source Apache RocketMQ cluster must meet the following requirements. If it does not, submit a ticket.

Destination instance limits

For the full list, see Quotas and limits.

SDK compatibility

The Remoting SDK uses the following Maven dependency and endpoint format:

<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>{version}</version>
</dependency>

producer.setNamesrvAddr("xxx:9876");
consumer.setNamesrvAddr("xxx:9876");

Scheduled and retry messages

Check whether your source cluster contains scheduled messages or messages pending retry. After migration, all producer and consumer connections switch to the ApsaraMQ for RocketMQ instance. Scheduled messages and retry messages that remain in the source cluster may not be consumed. Keep some consumer processes running in the source cluster to drain these messages before you decommission it.

Plan your migration batches

The migration tool supports per-topic migration with batch operations, canary release, and rollback, reducing the blast radius of each change.

1. Select topics and plan batches. Identify the topics to migrate and group them into batches. Start with non-critical business topics.

2. Coordinate with upstream and downstream teams. Notify all producer and consumer applications that use the selected topics. Every application must update its endpoint.

What the migration tool accesses

The migration tool accesses only the following information from the source cluster (least-privilege):

• Topic metadata

• Consumer group metadata

• Dynamic routing registration for topics

• Consumer connection and message accumulation status

The tool does not access other cluster data or write any configuration to the source cluster.

Procedure

1. Log on to the ApsaraMQ for RocketMQ console.

2. In the top navigation bar, select the region where both the source cluster and destination instance are located. In the left-side navigation pane, choose RocketMQ Copilot > Migration to Cloud.

3. On the Migration to Cloud page, click Create Task.

4. In the Create Migration Task panel, configure the parameters described in the following table and click OK.

5. In the Network Settings step of the migration wizard, enter the network details for the source Apache RocketMQ cluster and click Configure Network.

6. Wait for the network configuration to complete, then click Next.

Parameter reference

Procedure

1. In the Metadata Migration step, click the Topic Metadata tab.

2. Select a topic from the list, choose the correct Message Type from the drop-down list, and click Confirm and Import. To import multiple topics at once, select them and click Batch Import.

   Important

   Apache RocketMQ 4.x does not define message types at the topic level. ApsaraMQ for RocketMQ validates that the message type you assign matches the actual messages in the topic. Select the type based on your application logic. If you select an incorrect type, messages fail to send or receive after migration. If you are unsure about the message type, submit a ticket.

3. Click the Group Metadata tab. Select a consumer group, choose the correct Consumption Order from the drop-down list, and click Confirm and Import. To import multiple groups at once, select them and click Batch Import.

   Important

   In the Apache RocketMQ 4.x SDK, message consumption order is specified on the client side. In ApsaraMQ for RocketMQ 5.x, the broker controls consumption order at the group level. Select the order based on your application requirements. An incorrect order causes consumption exceptions. If you are unsure, submit a ticket.

4. Verify that all topics and groups for this migration batch are imported, then click Next.

Before you proceed

• After changing the endpoint, restart each application to connect it to the destination instance.

• Make sure every producer and consumer application updates its endpoint:

  • Producers that do not switch: some messages fail to send.

  • Consumers that do not switch: messages accumulate and cannot be consumed.

Endpoint change examples

producer.setNamesrvAddr("192.168.XX.XX:9876");
consumer.setNamesrvAddr("192.168.XX.XX:9876");

producer.setNamesrvAddr("rmq-cn-pe334******-vpc.cn-hangzhou.rmq.aliyuncs.com:8080");
// vipChannelEnabled defaults to false. If set to true, comment out this line.
// producer.setVipChannelEnabled(false);

consumer.setNamesrvAddr("rmq-cn-pe334******-vpc.cn-hangzhou.rmq.aliyuncs.com:8080");
// vipChannelEnabled defaults to false. If set to true, comment out this line.
// consumer.setVipChannelEnabled(false);

producer.setNamesrvAddr("rmq-cn-pe334******-vpc.cn-hangzhou.rmq.aliyuncs.com:8080");
// vipChannelEnabled defaults to true. Set it to false.
producer.setVipChannelEnabled(false);

consumer.setNamesrvAddr("rmq-cn-pe334******-vpc.cn-hangzhou.rmq.aliyuncs.com:8080");
// vipChannelEnabled defaults to true. Set it to false.
consumer.setVipChannelEnabled(false);

ClientConfiguration clientConfiguration = ClientConfiguration.newBuilder()
            .setEndpoints("192.168.XX.XX:9876")
            .setCredentialProvider(sessionCredentialsProvider)
            .build();

ClientConfiguration clientConfiguration = ClientConfiguration.newBuilder()
            .setEndpoints("rmq-cn-pe334******-vpc.cn-hangzhou.rmq.aliyuncs.com:8080")
            .setCredentialProvider(sessionCredentialsProvider)
            .build();

Procedure

After all applications have updated their endpoint and restarted, click Next in the Change Endpoint step.

Before you proceed

• At each stage, verify that messages are sent and received correctly before proceeding. If an issue occurs, roll back to the previous stage and troubleshoot.

• Switch traffic for all migrated topics and confirm normal operation before completing the migration task. Traffic switching settings cannot be modified after the task is marked complete.

Traffic switching stages

Each topic progresses through the following four stages:

Traffic topology for each stage:

Stage	Topology
Read and write in source cluster
Write in source cluster, read in both clusters
Write in destination cluster, read in both clusters
Read and write in destination cluster

Verification checks at each stage

Procedure

1. In the Message Migration step, select the topic to migrate and check its verification status.

   • Check Passed: Proceed to the next step.

   • Not passed: Troubleshoot based on the check result, then click Re-verify until the check passes.

   • Not passed, but confirmed non-blocking: Click Ignore Check and proceed.

2. In the Actions column, click Switch Traffic.

3. In the confirmation dialog, click OK.

4. Repeat steps 1 through 3 for each stage until Traffic Switching Stage shows Read and Write in Destination Cluster.

5. After all topics are fully switched, click Migrated at the bottom of the page.

Additional operations

The following operations are available on the Message Migration page:

• Roll Back

  • Roll back to the previous stage: Reverts the topic to the last stage where migration was working correctly. Troubleshoot before proceeding again.

  • Roll back to the initial stage: Reverts to the original routing state. Use this for emergency troubleshooting only.

    Note

    Rolling back to the initial stage may cause unconsumed messages produced during earlier stages to be delayed or cannot be processed.

• Create Topic: If a topic was not selected during metadata migration, create it directly on the destination ApsaraMQ for RocketMQ 5.x instance.

• Batch Traffic Switching / Batch Rollback: Switch traffic or roll back for multiple topics at once. Both operations apply only to topics at the same traffic switching stage.