Overview of best practices for server data collection - Elasticsearch

When applications are running, various types of data are generated, such as log data, system metric data, audit framework data, detection status data, and application performance monitoring (APM) data. You can select a solution based on your business requirements and business environment to collect the desired data and transfer the collected data to Alibaba Cloud Elasticsearch. This topic provides an overview of best practices for server data collection.


Best practice	Description
Data collection for Alibaba Cloud Elasticsearch	You can use one of the following tools to collect data: Beats, Logstash, clients, and Kibana.
Use Filebeat to collect Apache log data	You can use Filebeat to collect log data and use Logstash to filter the collected data and transfer the processed data to Elasticsearch for analysis.
Use Metricbeat to collect system data and NGINX service data	You can use Metricbeat to collect system data and NGINX service data and generate visual charts.
Use Auditbeat to collect system audit data and monitor file changes	You can use Auditbeat to collect data from the Linux audit framework, monitor system file changes, and generate visual charts.
Use Heartbeat to check ICMP and HTTP services	You can use Heartbeat to detect the statuses of Internet Control Message Protocol (ICMP) and HTTP services and generate visual charts.
Use user-created Metricbeat to collect system metrics	You can use Metricbeat to collect the metrics of a machine and send the collected data to Elasticsearch. Then, you can use Kibana to analyze and display the data in charts.
Use SkyWalking to implement end-to-end monitoring on Elasticsearch	You can use SkyWalking to implement end-to-end monitoring on an Elasticsearch cluster and use Kibana to analyze and view the collected monitoring data.
Use Uptime to monitor Alibaba Cloud Elasticsearch clusters in real time	You can use Heartbeat to detect the statuses of the HTTP or HTTPS, TCP, and ICMP services and send the collected data to the Uptime application in Kibana. Then, the Uptime application monitors the availability and response time of applications and services in real time and reports errors before your business is affected.