Problem description
You use an ApsaraMQ for Kafka instance whose version is 2.2.0 or later. The consumer offsets of a group are retained even if the consumer offsets are expired. You can still view the consumer details of the topic on the Subscriptions tab of the Topic Details page in the ApsaraMQ for Kafka console.
The Consumer Offset Retention Period parameter specifies the maximum duration for which consumer offsets can be retained. The retention period of a consumer offset starts from the point in time when the consumer offset is committed. When the retention period reaches the maximum allowed duration, the system deletes the consumer offset. Each time a consumer offset is committed to a partition in a topic, the system resets the retention period. For more information, see Modify message configurations.
Possible cause
The group is active. In this case, the system does not delete the consumer offsets of the group, regardless of whether the retention period of the consumer offsets exceeds the maximum retention period. This is a mechanism of Apache Kafka.
A group that contains an active consumer thread is active.
For example, you create a group named TestGroup to subscribe to Topic A, and consumer offsets are committed. Then, your business requirements change. You configure the TestGroup group to unsubscribe from Topic A and subscribe to Topic B. In this case, the TestGroup group is active. The system does not delete the consumer offsets for Topic A and Topic B.
If you configure the TestGroup group to unsubscribe from Topic A and do not configure the group to subscribe to Topic B, the TestGroup group is inactive. In this case, the system deletes the consumer offsets whose retention period exceeds the maximum retention period.
If you do not want to wait for a consumer offset to expire and be deleted, you can use a new group instead of the TestGroup group to subscribe to Topic B. If you want to wait for a consumer offset to expire and be deleted, you can use the TestGroup group to subscribe to Topic B after the consumer offsets for Topic A are deleted.
Solutions
To delete expired consumer offsets, perform the following steps:
If you require the group to remain active, log on to the ApsaraMQ for Kafka console and manually reset the expired consumer offsets by partition. For more information, see Reset consumer offsets.
If the group can be disconnected from ApsaraMQ for Kafka, perform one of the following operations:
Disconnect the group and wait for the consumer offsets to expire. After the consumer offsets expire, the system automatically deletes the consumer offsets.
Migrate the active group to a new group. Then, deactivate the consumer threads in the original group and wait for the consumer offsets of the original group to expire. After the consumer offsets expire, the system automatically deletes the consumer offsets.