Symptoms
A consumer uses the main consumption thread to synchronize the pulled messages to the network file system (NFS) that is attached to the consumer. This decreases the message consumption speed of the consumer and blocks message processing.
Causes
- An NFS cannot run fast enough.
- An NFS is a shared network storage system that supports simultaneous access from multiple consumers. However, the consumers compete for resources. The more consumers involve, the lower performance the NFS can provide.
Solutions
- We recommend that you use two independent threads to consume and store messages. One is used for message pulling, and the other is used for message storage. After the thread for message pulling processes a message, the thread forwards the processing results to the other thread for storage and continues message consumption. This ensures consumption efficiency.
- You can also attach an ultra disk or a solid-state drive (SSD) to each consumer to independently store processing results. This prevents performance degradation due to the competition for NFS resources among consumers. To store the processing results of different consumers in the same NFS, you can use an asynchronous tool or thread to forward the processing results that are stored in cloud disks to the NFS. This prevents message processing from being blocked by synchronous storage in the NFS. Asynchronous processing is an effective way to prevent performance degradation due to competitive resource access.