ApsaraMQ for Kafka allows you to use the automatic topic creation feature in cloud migration and testing scenarios. After you enable the feature, you can create topics by writing code. This eliminates the need to manually create topics and improves efficiency.
Prerequisites
An ApsaraMQ for Kafka instance that meets the following conditions is purchased and deployed.
Item | Description |
Status | Running |
Version | Major version: V2.2.0 or later. Minor version: latest. |
You can view the status and version information of an instance in the Basic Information section of the Instance Details page in the ApsaraMQ for Kafka console.
Usage notes
The automatic topic creation feature is suitable for the cloud migration and testing scenarios. We recommend that you do not enable the feature in production environments. If you enable the feature in production environments, do not use it in major production activities.
After you enable the access control list (ACL) feature for a Professional Edition instance, a topic is not automatically created if you send a message to the instance without specifying a topic.
Procedure
Log on to the ApsaraMQ for Kafka console. In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.
On the Instances page, click the name of the instance that you want to manage.
In the Configuration Information section of the Instance Details page, turn on the Automatic Topic Creation switch.
NoteThe following items describe the naming conventions for topics that are created by using the automatic topic creation feature:
The name must be 3 to 64 characters in length.
The name can contain only letters, digits, hyphens (-), and underscores (_), and must contain at least one letter or digit.
After you enable the automatic topic creation feature, a topic can be created by sending a message to your ApsaraMQ for Kafka Standard Edition instance without specifying a topic.
FAQ
Can topics be automatically created?
Yes, topics can be automatically created. However, we recommend that you do not enable the automatic topic creation feature. ApsaraMQ for Kafka allows you to manage topics in the console or by calling API operations. We recommend that you do not enable the automatic topic creation feature in production environments due to the following reasons:
If you enable the automatic topic creation feature, features such as fine-grained permission control and resource action audit cannot be used. This poses security risks to your business.
If you enable the automatic topic creation feature, you may encounter issues such as excess resource usage and inefficient resource management. These issues may lead to system instability.
A topic may fail to be automatically created even if you enable the automatic topic creation feature. This may affect your business. For more information, see What do I do if topics fail to be automatically created?
In what scenarios can I enable the automatic topic creation feature?
We recommend that you do not enable the automatic topic creation feature except for in scenarios such as cloud migration and testing. After you enable the feature, we recommend that you use it only in the migration and testing scenarios.
What do I do if topics fail to be automatically created?
You can use the following methods to resolve the issue:
If a moderate number of topics fail to be automatically created, you can manually create them in the ApsaraMQ for Kafka console or by calling the corresponding API operation.
If a large number of topics fail to be automatically created, you can troubleshoot the issue by checking the logs or referring to the following possible reasons:
Invalid operations: If you perform invalid operations, topics fail to be automatically created and cannot be used. This may affect your business. The following items describe possible invalid operations:
Topic name: After you enable the automatic topic creation feature, the system automatically creates topics when necessary. If the name of a topic does not follow the naming conventions, the topic cannot be created. For example, the name is invalid or the length of the name exceeds the limit.
Topic settings: If you specify an improper engine for topics or the remaining quotas of topics and partitions are insufficient, topics fail to be automatically created.
ACL: After the ACL feature is enabled, a topic is not automatically created when you send a message to your ApsaraMQ for Kafka instance without specifying a topic.
Unexpected factors: Unexpected factors may arise and cause significant delays in the automatic creation of topics. In this case, the topics fail to be created because the creation request times out. This seriously affects your business. These factors include network jitters, broker upgrades, console upgrades or maintenance, and Elastic Compute Service (ECS) instance failures.
Data conflicts: When the system automatically creates a topic, the system may adopt a name that is the same as another topic. In this case, the data of the two topics may be both recorded and the system may fail to detect the conflict. Even if the conflict is detected, it can rarely be resolved and may lead to irreversible losses.
Latency: Triggering a topic that is automatically created requires much longer time than triggering an existing topic. This may cause temporary business interruptions. In most cases, business cannot afford such interruptions.
Insufficient resources: If the upper limit of partitions or topics is reached, no more topics can be created.
References
For information about how to migrate a self-managed or cross-cloud instance to an ApsaraMQ for Kafka instance, see Migrate a self-managed or cross-cloud instance to an ApsaraMQ for Kafka instance.
For information about how to create a topic in the ApsaraMQ for Kafka console or by calling API operations, see Step 3: Create resources and CreateTopic.