Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Bolstering Your Cluster Stability with Alibaba Cloud Elasticsearch Aliyun-qos Plugin

Bolstering Your Cluster Stability with Alibaba Cloud Elasticsearch Aliyun-qos Plugin

In this article, we'll explore how to effectively leverage the aliyun-qos plugin on Alibaba Cloud Elasticsearch for superior performance tuning through read-write throttling mechanisms.

Elasticsearch is widely used for indexing and searching large datasets with blazing fast speed and high efficiency. However, performance challenges may arise as your data grows or query load increases. Sustaining optimal performance demands advanced solutions like the Alibaba Cloud Elasticsearch aliyun-qos plugin--a throttle management system designed to bolster cluster stability under diverse workloads.

In this article, we'll explore how to effectively leverage the aliyun-qos plugin on Alibaba Cloud Elasticsearch for superior performance tuning through read-write throttling mechanisms. Let's dive into the technicalities with some hands-on code examples.

Prerequisites for Using aliyun-qos Plugin

Before proceeding, ensure the following:

  • The aliyun-qos plugin is upgraded to the latest version on your Elasticsearch cluster. Use the Kibana console to check the plugin version with the command:
GET /_cat/plugins?v

The latest version for an Elasticsearch V7.10 cluster is 7.10.0_ali1.6.0.2. For other versions, the format is -rc4. Follow Alibaba Cloud's upgrade instructions if necessary.
Note: An unsupported_operation_exception error occurs for plugin versions earlier than rc4.

Evaluating Throttling Thresholds

Establishing the right throttling thresholds is crucial to prevent performance bottlenecks. Calculate the throttling thresholds considering client QPS to Elasticsearch and account for replica shards for write requests.
For query requests:

Throttling threshold for query requests = End-to-end QPS from client to Elasticsearch

For write requests:
Adjust the threshold based on the number of replica shards. Include additional write traffic from tasks like X-Pack Monitor, Audit, and Watcher.

Enabling and Disabling Throttling

The aliyun-qos plugin remains disabled by default. To enable or disable throttling, use the code blocks below:
For enabling throttling: Elasticsearch V7.10 cluster:

PUT _cluster/settings
{
  "persistent": {
    "apack.qos.limiter.enabled": true
  }
}

For other versions:

PUT _cluster/settings
{
   "persistent": {
      "apack.qos.ratelimit.enabled": "true"
   }
}

For disabling throttling:
This can be done by setting the parameter to false or null:

PUT _cluster/settings
{
  "persistent": {
    "apack.qos.limiter.enabled": false
  }
}

Configure a Limiter

To configure a limiter for throttling, use the following example:

PUT /_qos/limiter/<limiterName>
{
  "limiters": {
    "search.qps": "1000"
  },
  "tags": {
    "index": "nginx-log-*"
  }
}

Adjust the limiterName, limiters, and tags values as needed. Replace nginx-log-* with the relevant index pattern for your use case.

Querying and Deleting Limiters

To query all limiters:

GET _qos/limiter

For a specific limiter:

GET _qos/limiter/<limiterName>

To delete limiters:

DELETE _qos/limiter/<limiterName>

Obtaining Monitoring Data

To monitor throttling metrics:
For current data:

GET /_qos/limiter/nodes/stats
GET /_qos/limiter/nodes/{nodeId}/stats
GET /_qos/limiter/nodes/{nodeId}/stats/{limiterIds}

For historical data:

GET /_qos/limiter/metric
GET /_qos/limiter/metric/{limiterId}

Remember that achieving optimal performance requires fine-tuning with real-world workloads. These code examples provide a solid starting point for improving cluster stability through the aliyun-qos plugin on Alibaba Cloud Elasticsearch.

Conclusion

With the right–albeit complex–configuration of the aliyun-qos plugin, Alibaba Cloud Elasticsearch can manage heavy read-write operations efficiently while ensuring stable performance. These adjustments are indispensable for clusters operating under variable loads.
Ready to start your journey with Elasticsearch on Alibaba Cloud? Explore our tailored Cloud solutions and services to take the first step towards transforming your data into a visual masterpiece.

Please Click here, Embark on Your 30-Day Free Trial

Elasticsearch Cloud Elasticsearch Plugin Elasticsearch Cluster Stability
0 1 0
Share on

Read previous post:

Leveraging the Power of the Analysis-ALIWS Plugin on Alibaba Cloud Elasticsearch

Read next post:

How to Migrating Your Data from a Self-Managed Elasticsearch to Alibaba Cloud Elasticsearch with OSS

Data Geek

100 posts | 4 followers

You may also like

Comments

Data Geek

100 posts | 4 followers

Related Products

  • Alibaba Cloud Elasticsearch

    Alibaba Cloud Elasticsearch helps users easy to build AI-powered search applications seamlessly integrated with large language models, and featuring for the enterprise: robust access control, security monitoring, and automatic updates.

    Learn More

  • Hybrid Cloud Solution

    Highly reliable and secure deployment solutions for enterprises to fully experience the unique benefits of the hybrid cloud

    Learn More

  • CloudBox

    Fully managed, locally deployed Alibaba Cloud infrastructure and services with consistent user experience and management APIs with Alibaba Cloud public cloud.

    Learn More

  • FinTech on Cloud Solution

    This solution enables FinTech companies to run workloads on the cloud, bringing greater customer satisfaction with lower latency and higher scalability.

    Learn More

More Posts by Data Geek

See All