X-Pack Watcher is a monitoring and alerting service that is developed for Elasticsearch. If you configure X-Pack Watcher for your Elasticsearch cluster, X-Pack Watcher can trigger actions when specific conditions are met. For example, if the logs index contains errors, X-Pack Watcher triggers the system to send alert notifications by using emails or WeCom messages. This topic describes how to configure a WeCom chatbot to receive alert notifications from X-Pack Watcher.
Prerequisites
An Alibaba Cloud Elasticsearch cluster is created. For more information, see Create an Alibaba Cloud Elasticsearch cluster.
NoteIn the original network architecture, X-Pack Watcher is available only for single-zone Elasticsearch clusters.
In the new network architecture, you need to configure private connections for Elasticsearch clusters to connect the clusters to the Internet. For more information, see Configure a private connection for an Elasticsearch cluster.
For more information about the network architecture, see [Notice] Network architecture adjustment.
X-Pack Watcher is enabled for the Elasticsearch cluster. By default, X-Pack Watcher is disabled. For more information, see Configure the YML file.
An Elastic Compute Service (ECS) instance is created in your virtual private cloud (VPC). For more information, see Create an instance by using the wizard.
NoteX-Pack Watcher cannot directly access the Internet and must use the internal endpoint of your Elasticsearch cluster to access the Internet. In this case, you can enable source network address translation (SNAT) for or associate an elastic IP address (EIP) with an ECS instance that is deployed in your VPC. This way, you can use the ECS instance as a proxy to forward requests. For more information, see Associate an EIP or Configure SNAT.
A WeCom chatbot is configured, and the webhook URL of the chatbot is obtained.
Procedure
Step 1: Configure an NGINX proxy and configure a security group rule for the ECS instance
Configure an NGINX proxy on the ECS instance.
The NGINX proxy is used to forward alerts notifications. X-Pack Watcher sends alert notifications to the proxy, which then forwards the notifications to DingTalk or WeCom.
Install NGINX on the ECS instance.
Configure the nginx.conf file.
Replace the
server
configuration in the nginx.conf file with the following code:server { listen 8080; server_name _; root /usr/share/nginx/html; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { proxy_pass <Webhook URL of the WeCom chatbot>; } error_page 404 /404.html; location = /40x.html { } error_page 500 502 503 504 /50x.html; location = /50x.html { } }
Replace <Webhook URL of the WeCom chatbot> with the webhook URL of the WeCom chatbot that you configured to receive alert notifications.
Reload the NGINX configuration file and restart NGINX.
/usr/local/webserver/nginx/sbin/nginx -s reload # Reload the NGINX configuration file. /usr/local/webserver/nginx/sbin/nginx -s reopen # Restart NGINX.
Configure a security group rule for the ECS instance.
The security group rule is used to allow the NGINX proxy on the ECS instance to receive alert notifications from the Elasticsearch cluster.
Log on to the ECS console.
In the left-side navigation pane, choose Instances & Images > Instances.
On the Instances page, find the ECS instance and click its name.
On the instance details page, click the Security Groups tab.
On the Security Groups tab, click the name of the desired security group.
On the Inbound tab of the Access Rule section, click Add Rule.
Configure the parameters.
Parameter
Description
Action
Select Allow.
Priority
Retain the default value.
Protocol Type
Select Custom TCP.
Port Range
Set this parameter to the port that you frequently use. In this example, port 8080 is used.
Authorization Object
Enter the IP addresses of all nodes in the Elasticsearch cluster.
NoteFor more information about how to obtain the IP addresses of the nodes, see View the basic information of nodes.
Description
The description of the rule.
Click Save.
Step 2: Configure a watch for alerting
- Log on to the Kibana console of your Elasticsearch cluster and go to the homepage of the Kibana console as prompted. For more information about how to log on to the Kibana console, see Log on to the Kibana console.Note In this example, an Elasticsearch V6.7.0 cluster is used. Operations on clusters of other versions may differ. The actual operations in the console prevail.
- In the left-side navigation pane of the page that appears, click Dev Tools.
On the Console tab of the page that appears, run the following command to create a watch.
In this example, a watch named developer_count_watch is created to search the zl-testgaes index for the developer field every 10 seconds. If the value of the developer field is Nintendo and the number of occurrences of the developer field is more than 158,974, an alert is triggered.
PUT _xpack/watcher/watch/developer_count_watch { "trigger": { "schedule": { "interval": "10s" } }, "input": { "search": { "request": { "indices": ["zl-testgaes"], "body": { "query": { "bool": { "must": [ {"match": { "developer" : "Nintendo" } }, { "range": { "year_of_release": { "gte": "2011-09-20T16:00:00.000Z", "lte": "2011-12-31T16:00:00.000Z" } } } ] } } } } } }, "condition": { "compare": { "ctx.payload.hits.total": { "gt": 158974 } } }, "actions" : { "test_issue" : { "webhook" : { "method" : "POST", "url" : "http://<yourAddress>:8080", "body" : "{\"msgtype\": \"text\", \"text\": { \"content\": \"developer is Nintendo, More than 158974\"}}" } } } }
Table 1. Parameters
Parameter
Network architecture
Description
url
New network architecture
The domain name of the endpoint. Requests are forwarded based on the domain name. For more information about how to obtain the domain name of an endpoint, see Configure a private connection for an Elasticsearch cluster.
Original network architecture
Set this parameter to one of the following items:
IP address of the NGINX proxy. In this case, requests are forwarded over the Internet by using the NGINX proxy that resides in the same VPC as the Elasticsearch cluster.
The webhook URL of the WeCom chatbot.
NoteIf the error
No handler found for uri [/_xpack/watcher/watch/log_error_watch_2] and method [PUT]
is returned after you run the preceding command, X-Pack Watcher is disabled for the Elasticsearch cluster. In this case, enable X-Pack Watcher and run the command again. For more information, see Configure the YML file.
Step 3: View the alert notifications
In normal cases, if the conditions specified in Step 2: Configure a watch for alerting are met, the alert notifications are sent to your WeCom group, as shown in the following figure.
If you no longer require this watch, you can run the following command to delete the watch:
DELETE _xpack/watcher/watch/developer_count_watch