Bolstering Your Cluster Stability with Alibaba Cloud Elasticsearch Aliyun-qos Plugin

In this article, we'll explore how to effectively leverage the aliyun-qos plugin on Alibaba Cloud Elasticsearch for superior performance tuning through read-write throttling mechanisms.

Elasticsearch is widely used for indexing and searching large datasets with blazing fast speed and high efficiency. However, performance challenges may arise as your data grows or query load increases. Sustaining optimal performance demands advanced solutions like the Alibaba Cloud Elasticsearch aliyun-qos plugin--a throttle management system designed to bolster cluster stability under diverse workloads.

In this article, we'll explore how to effectively leverage the aliyun-qos plugin on Alibaba Cloud Elasticsearch for superior performance tuning through read-write throttling mechanisms. Let's dive into the technicalities with some hands-on code examples.

Prerequisites for Using aliyun-qos Plugin

Before proceeding, ensure the following:

The aliyun-qos plugin is upgraded to the latest version on your Elasticsearch cluster. Use the Kibana console to check the plugin version with the command:

GET /_cat/plugins?v

The latest version for an Elasticsearch V7.10 cluster is 7.10.0_ali1.6.0.2. For other versions, the format is -rc4. Follow Alibaba Cloud's upgrade instructions if necessary.
Note: An unsupported_operation_exception error occurs for plugin versions earlier than rc4.

Evaluating Throttling Thresholds

Establishing the right throttling thresholds is crucial to prevent performance bottlenecks. Calculate the throttling thresholds considering client QPS to Elasticsearch and account for replica shards for write requests.
For query requests:

Throttling threshold for query requests = End-to-end QPS from client to Elasticsearch

For write requests:
Adjust the threshold based on the number of replica shards. Include additional write traffic from tasks like X-Pack Monitor, Audit, and Watcher.

Enabling and Disabling Throttling

The aliyun-qos plugin remains disabled by default. To enable or disable throttling, use the code blocks below:
For enabling throttling: Elasticsearch V7.10 cluster:

PUT _cluster/settings
{
  "persistent": {
    "apack.qos.limiter.enabled": true
  }
}

For other versions:

PUT _cluster/settings
{
   "persistent": {
      "apack.qos.ratelimit.enabled": "true"
   }
}

For disabling throttling:
This can be done by setting the parameter to false or null:

PUT _cluster/settings
{
  "persistent": {
    "apack.qos.limiter.enabled": false
  }
}

Configure a Limiter

To configure a limiter for throttling, use the following example:

PUT /_qos/limiter/<limiterName>
{
  "limiters": {
    "search.qps": "1000"
  },
  "tags": {
    "index": "nginx-log-*"
  }
}

Adjust the limiterName, limiters, and tags values as needed. Replace nginx-log-* with the relevant index pattern for your use case.

Querying and Deleting Limiters

To query all limiters:

GET _qos/limiter

For a specific limiter:

GET _qos/limiter/<limiterName>

To delete limiters:

DELETE _qos/limiter/<limiterName>

Obtaining Monitoring Data

To monitor throttling metrics:
For current data:

GET /_qos/limiter/nodes/stats
GET /_qos/limiter/nodes/{nodeId}/stats
GET /_qos/limiter/nodes/{nodeId}/stats/{limiterIds}

For historical data:

GET /_qos/limiter/metric
GET /_qos/limiter/metric/{limiterId}

Remember that achieving optimal performance requires fine-tuning with real-world workloads. These code examples provide a solid starting point for improving cluster stability through the aliyun-qos plugin on Alibaba Cloud Elasticsearch.

Conclusion

With the right–albeit complex–configuration of the aliyun-qos plugin, Alibaba Cloud Elasticsearch can manage heavy read-write operations efficiently while ensuring stable performance. These adjustments are indispensable for clusters operating under variable loads.
Ready to start your journey with Elasticsearch on Alibaba Cloud? Explore our tailored Cloud solutions and services to take the first step towards transforming your data into a visual masterpiece.

Please Click here, Embark on Your 30-Day Free Trial

Community

Bolstering Your Cluster Stability with Alibaba Cloud Elasticsearch Aliyun-qos Plugin

Prerequisites for Using aliyun-qos Plugin

Evaluating Throttling Thresholds

Enabling and Disabling Throttling

Configure a Limiter

Querying and Deleting Limiters

Obtaining Monitoring Data

Conclusion

Read previous post:

Read next post:

Data Geek

You may also like

Comments

Data Geek

Related Products

Alibaba Cloud Elasticsearch

Hybrid Cloud Solution

CloudBox

FinTech on Cloud Solution