When you create a topic on an ApsaraMQ for Kafka instance, you can specify cloud storage or local storage as the storage engine. Cloud storage uses the Alibaba Cloud disk algorithm and features low latency and high auto scaling capabilities. This storage engine is suitable for most scenarios. If you have special requirements on specific features, such as compaction, idempotence, transactions, and partitionally ordered messages, we recommend that you use local storage.
Comparison between cloud storage and local storage
For information about the Apache Kafka versions that different editions of ApsaraMQ for Kafka support, see Instance editions.
When you create a topic on a Professional Edition instance, you can set the Storage Engine parameter to Cloud Storage or Local Storage. When you create a topic on a Standard Edition instance, the Storage Engine parameter is unavailable.
Local storage does not refer to local disks. Local storage refers to the use of the native In-Sync Replicas (ISR) algorithm.
Item | Cloud storage | Local storage |
Underlying algorithm | Alibaba Cloud disk algorithm | ISR algorithm of native Apache Kafka |
Requirements on instance edition | None | |
Replica mechanism | Three replicas in distributed mode
| Three replicas in distributed mode If you purchase a disk with 300 GB of storage space, you can store 100 GB of business data on the disk. The remaining 200 GB of storage space is used to store the two replicas of your business data. |
Message sending latency | Low | High |
Partition consumption ratio | 1:1 If you create a topic that requires 12 partitions, 12 partitions are deducted from the partition quota of the instance that contains the topic. | 1:3 If you create a topic that requires 12 partitions, 36 partitions are deducted from the partition quota of the instance that contains the topic. |
Fragmented storage | No | Yes |
Compaction | Not supported | Supported |
Idempotence | Not supported | Supported |
Transaction | Not supported | Supported |
Message type | Normal messages If restarts or failures occur in a cluster, a small number of messages in the cluster may be out of order. | Partitionally ordered messages If failures occur in a cluster, the system automatically selects a new leader from the ISR. |
References
For information about how to select a storage engine for a topic that is created on an ApsaraMQ for Kafka Professional Edition instance, see Step 1: Create a topic.