ApsaraMQ for Kafka:FAQ about automatic topic creation

Yes, topics can be automatically created. However, we recommend that you do not enable the automatic topic creation feature. ApsaraMQ for Kafka allows you to manage topics in the console or by calling API operations. By default, you cannot use the automatic topic creation feature. We recommend you do not enable this feature due to the following reasons:

• If you enable the automatic topic creation feature, features such as fine-grained permission control and resource action audit cannot be used. This poses security risks to your business.

• If you enable the automatic topic creation feature, you may encounter issues such as excess resource usage and inefficient resource management. These issues may lead to system instability.

• A topic may fail to be automatically created even if you enable the automatic topic creation feature. This may affect your business. For more information, see What do I do if topics fail to be automatically created?

You can use the following methods to resolve the issue:

• If a moderate number of topics fail to be automatically created, you can manually create the topics in the ApsaraMQ for Kafka console or by calling the corresponding API operation.

• If a large number of topics fail to be automatically created, you can troubleshoot the issue by checking the logs or based on the following reasons:

  • Invalid operations: If you perform invalid operations, topics fail to be automatically created and cannot be used. This may affect your business. The following items describe the details:

    • Topic names: After you enable the automatic topic creation feature, the system automatically creates topics when necessary. If the name of a topic does not follow the naming conventions, the topic cannot be created. For example, the name is invalid or the length of the name exceeds the limit.

    • Invalid configurations: If you specify an improper engine for topics or the remaining quotas of topics and partitions are insufficient, topics fail to be automatically created.

  • Unexpected factors: Unexpected factors may arise and cause significant delays in the automatic creation of topics. In this case, topics fail to be created because creation times out. This seriously affects your business. These factors include network jitters, server upgrades, console upgrades and maintenance, and Elastic Compute Service (ECS) instance failures.

  • Data conflicts: When the system automatically creates a topic, the system may adopt a name that is the same as another topic. If this happens, the automatic creation fails. However, the data of the two topics may be both recorded and the system may fail to identify this conflict. Such a conflict can rarely be resolved and may lead to irreversible losses.

  • Latency: Triggering a topic that is automatically created requires much longer time than triggering an existing topic. This may cause temporary business interruptions. In most cases, business cannot afford such interruptions.

  • Insufficient resources: If the upper limit of partitions or topics is reached, no more topics can be created.

  • Other reasons.