Migrate topics from a self-managed Kafka cluster to ApsaraMQ for Kafka - ApsaraMQ for Kafka

This topic describes how to use the migration tool provided by ApsaraMQ for Kafka to migrate topics from a self-managed Kafka cluster to a ApsaraMQ for Kafka instance.

Prerequisites

Make sure that the following operations are complete:

Java Development Kit (JDK) 8 is downloaded and installed. For more information, see Java Downloads.
The migration tool is downloaded and installed. If the migration tool is not downloaded, click kafka-migration-assessment.jar to download the migration tool.
A ApsaraMQ for Kafka instance is purchased and deployed.
- For information about how to purchase an instance and deploy the instance in a virtual private cloud (VPC), see Purchase and deploy an instance that allows access from a VPC.
- For information about how to purchase an instance that can be accessed over the Internet and a VPC and deploy the instance, see Purchase and deploy an instance that allows access from the Internet and a VPC.

Procedure

Open your CLI.
Run the cd command to go to the directory where the migration tool is stored.

Run the following command to check the topics that you want to migrate:

java -jar kafka-migration-assessment.jar TopicMigrationFromZk  \ 
--sourceZkConnect 192.168.XX.XX  \ 
--destAk <yourdestAccessKeyId>  \ 
--destSk <yourdestAccessKeySecret>  \ 
--destRegionId <yourdestRegionId>  \ 
--destInstanceId <yourdestInstanceId>

Parameter	Description
sourceZkConnect	The IP address of the source Apache Kafka cluster in ZooKeeper.
destAk	The AccessKey ID of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs.
destSk	The AccessKey secret of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs.
destRegionId	The ID of the region where the destination ApsaraMQ for Kafka instance is deployed.
destInstanceId	The ID of the destination ApsaraMQ for Kafka instance.

Check the information that is returned. The returned information describes the topics that are migrated.

13:40:08 INFO - Begin to migrate topics:[test]
13:40:08 INFO - Total topic number:1
13:40:08 INFO - Will create topic:test, isCompactTopic:false, partition number:1

Run the following command to commit the migration task:

java -jar kafka-migration-assessment.jar TopicMigrationFromZk  \ 
--sourceZkConnect 192.168.XX.XX  \ 
--destAk <yourdestAccessKeyId>  \ 
--destSk <yourdestAccessKeySecret>  \ 
--destRegionId <yourdestRegionId>  \ 
--destInstanceId <yourdestInstanceId>  \
--commit

Parameter	Description
commit	Commits the migration task.

After you commit the migration task, the following information is returned:

13:51:12 INFO - Begin to migrate topics:[test]
13:51:12 INFO - Total topic number:1
13:51:13 INFO - cmd=TopicMigrationFromZk, request=null, response={"code":200,"requestId":"7F76C7D7-AAB5-4E29-B49B-CD6F1E0F508B","success":true,"message":"operation success"}
13:51:13 INFO - TopicCreate success, topic=test, partition number=1, isCompactTopic=false

Perform the following steps to check whether a topic is migrated:
1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.
3. On the Instances page, click the name of the instance that you want to manage.
4. In the left-side navigation pane, click Topics.
5. On the Topics page, check whether the topic is displayed.