All Products
Search
Document Center

Elasticsearch:Overview of log synchronization and analysis

Last Updated:Aug 22, 2023

When applications are running, various types of logs are generated. You can collect the desired log data and transfer the collected data to Alibaba Cloud Elasticsearch. Then, you can query and analyze the data. This topic provides solutions for log synchronization and analysis.

Solution

Description

Use Filebeat to collect Apache log data

The typical log collection mode of Elastic Stack is used. Use Alibaba Cloud Filebeat to collect Apache log data. Then, use Alibaba Cloud Logstash to filter the collected data and transfer the processed data to an Alibaba Cloud Elasticsearch cluster for queries and analysis.

Use the logstash-input-sls plug-in to obtain logs from Simple Log Service

Use the logstash-input-sls plug-in to obtain logs from Simple Log Service and transfer them to Alibaba Cloud Elasticsearch for queries and analysis.

Use self-managed Filebeat to collect MySQL logs

Use self-managed Filebeat to collect and send MySQL logs to Alibaba Cloud Elasticsearch. Then, query, analyze, and present these logs in the Kibana console in a visualized manner.

Use Alibaba Cloud Elasticsearch and Rsbeat to analyze Redis slow logs in real time

Use Rsbeat to collect and send Redis slow logs to Alibaba Cloud Elasticsearch. Then, perform graphical analysis on the logs in the Kibana console.

Use a Fleet agent to collect NGINX log data

Use a Fleet agent to collect NGINX log data from a server and analyze the collected data in a visualized manner in the Kibana console. This can help improve the network performance and security of the NGINX server.

Use a Fleet agent to collect NetFlow log data

NetFlow is a network traffic analysis technology that is used to monitor, analyze, and diagnose network traffic. You can use a Fleet agent to collect NetFlow log data from a server. This can help you monitor network traffic of a server in real time and analyze the network traffic.

Use a Fleet agent to collect custom log data

Use a Fleet agent to collect custom log data from a server. This can help you monitor and analyze data based on your business scenario.