Use Metricbeat to collect the metrics of a machine - Elasticsearch

If you want to view and analyze the metrics of a machine, you can use Metricbeat to collect the metrics. Then, Metricbeat sends the collected data to an Alibaba Cloud Elasticsearch cluster. You can search for and analyze the data and generate a dashboard for the data in the Kibana console of the cluster. This topic uses a computer that runs the macOS operating system to describe the detailed procedure.

Prerequisites

The following operations are performed:

Create an Alibaba Cloud Elasticsearch cluster.
For more information, see Create an Alibaba Cloud Elasticsearch cluster.
Note
If you want to access the Elasticsearch cluster by using its internal endpoint, you must purchase an Elastic Compute Service (ECS) instance that resides in the same virtual private cloud (VPC), region, and zone as the Elasticsearch cluster. For more information, see Create an instance on the Custom Launch tab.
Download Metricbeat.

Background information

Beats is a platform for single-purpose data shippers. After the data shippers are installed, they send data from thousands of machines and systems to Logstash or Elasticsearch.

Metricbeat is a lightweight data shipper that collects metrics from your systems and services. System metrics include CPU and memory metrics. Service metrics include Redis and NGINX metrics.

Procedure

Configure an Alibaba Cloud Elasticsearch cluster.
Configure Metricbeat.
View the related dashboard in the Kibana console.
Note
You can also use Metricbeat to collect metrics from a computer that runs the Linux or Windows operating system. Then, Metricbeat sends the metrics to Alibaba Cloud Elasticsearch.

Configure an Alibaba Cloud Elasticsearch cluster

Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, click Elasticsearch Clusters.
Navigate to the desired cluster.
1. In the top navigation bar, select the resource group to which the cluster belongs and the region where the cluster resides.
2. On the Elasticsearch Clusters page, find the cluster and click its ID.
In the left-side navigation pane of the page that appears, click Security.
On the page that appears, turn on the Public Network Access switch. After this switch turns green, click Modify next to Public IP Address Whitelist. In the Public IP Address Whitelist panel, add the public IP address of your computer running macOS to a whitelist.
Important
If you are using a public network such as Wi-Fi, add the IP address of the jump server that controls outbound traffic of the public network to a whitelist. If you cannot obtain this IP address, we recommend that you add 0.0.0.0/1,128.0.0.0/1 to a whitelist. These two CIDR blocks are used in this example. This configuration allows almost all public IP addresses to access the Elasticsearch cluster. We recommend that you evaluate the risks before you use this configuration.
In the left-side navigation pane, click Basic Information. In the Basic Information section of the page that appears, obtain the public endpoint of the Elasticsearch cluster for future use.
In the left-side navigation pane, click Cluster Configuration. In the YML File Configuration section of the page that appears, click Modify Configuration on the right side of YML File Configuration. In the YML File Configuration panel, set Auto Indexing to Enable.
Warning
This configuration takes effect only after the Elasticsearch cluster is restarted. To prevent impacts on your business, exercise caution when you change the setting of Auto Indexing.
Select This operation will restart the cluster. Continue? and click OK.
You can view the restart progress in the Tasks dialog box. After the Elasticsearch cluster is restarted, the configuration of the cluster takes effect.

Configure Metricbeat

Decompress the Metricbeat installation package that you have downloaded and go to the Metricbeat folder.

Open the metricbeat.yml file and edit the Elasticsearch output section in the file. You must uncomment the involved content.

编辑metricbeat.yml文件

Parameter	Description
hosts	The internal or public endpoint of the Elasticsearch cluster. In this example, the public endpoint of the Elasticsearch cluster is used.
protocol	Set this parameter to http.
username	The default value of this parameter is elastic.
password	The password that corresponds to the username. The password of the elastic account is specified when you create the Elasticsearch cluster. If you forget the password, you can reset it. For more information about the procedure and precautions for resetting the password, see Reset the access password of an Elasticsearch cluster.

Run the following command to start Metricbeat:
```
./metricbeat -e -c metricbeat.yml
```
After Metricbeat is started, it begins to send data to your Elasticsearch cluster.

View the related dashboard in the Kibana console

Log on to the Kibana console of the Elasticsearch cluster.
For more information, see Log on to the Kibana console.
In the left-side navigation pane, click Management and follow these steps to create an index pattern:
Important
If you have created an index pattern, skip this step.
1. In the Kibana section of the Management page, click Index Patterns.
2. In the Create index pattern section, enter an index pattern name (the name of the index that you want to query).
3. Click Next step.
4. Click Create index pattern.
In the left-side navigation pane, click Dashboard.
On the Dashboards page, you can view the collected data.
- The following figure shows relevant metrics.
- Click Metricbeat-cpu to view CPU metrics.
  Note
  You can configure metrics to be refreshed at 5-second intervals. The system generates reports for the metrics collected at these intervals. You can also connect to WebHook to configure alerts.