Alibaba Cloud Logstash is a server-side data processing pipeline. It allows you to dynamically collect data from a variety of sources, transform the data, and then save the data to a specific destination. Alibaba Cloud Logstash can process and transform all types of events by using input, output, and filter plug-ins. Alibaba Cloud Logstash is compatible with all the features of open source Logstash.
Benefits
Alibaba Cloud Logstash offers a plug-in center to meet your business requirements in a variety of scenarios. The plug-in center includes not only all open source built-in plug-ins but also self-developed plug-ins, such as logstash-input-sls, logstash-input-oss, and logstash-output-oss. The plug-in center is connected to the data ecosystem on the cloud and enables you to process and migrate data in an efficient manner.
Elasticsearch, Logstash, and Kibana form an ecosystem. Elasticsearch is a distributed, real-time search and analytics engine. Logstash allows you to collect, transform, process, and generate data. Kibana provides a visual interface for Elasticsearch. You can use Alibaba Cloud Elasticsearch to process logs, perform full-text searches, and analyze data in real time.
Data transmission mechanism
- Data collection: Logstash can collect data from a variety of sources, such as logs, metrics, applications, and storage, in streaming mode.
- Real-time parsing and data transformation: Logstash uses filters to parse all types of events and identifies defined fields to build schemas. Then, Logstash transforms the schemas into common data types and transmits data to a repository.
- Data storage and export: Logstash can send data to a specific destination. A variety of destinations are supported.
Features
| Feature | References |
|---|---|
| Supports fast deployment, easy management, simplified O&M, and flexible scale-out. | |
| Supports only deployment in virtual private clouds (VPCs). This improves service security. | Create a cluster |
| Allows you to upload third-party driver files and manage all driver files. | Configure third-party libraries |
| Provides an open, flexible plug-in center. The plug-in center is integrated with all open source input, output, and filter plug-ins and provides self-developed plug-ins, such as logstash-input-sls, logstash-input-oss, and logstash-output-oss. | Overview of plug-ins |
| Allows you to connect your Logstash cluster to the Internet by using a NAT gateway. | Configure a NAT gateway for data transmission over the Internet |
| Allows you to manage pipelines in a centralized manner by using the configuration file of your Logstash cluster. Allows you to view the output data of your pipeline in the console after you create and deploy the pipeline. |