Introduction to LoongCollector

LoongCollector is a next-generation collection agent that offers a range of benefits. This topic explores the concept and advantages of LoongCollector.

LoongCollector overview

LoongCollector is a data collector that offers high performance, robust stability, and flexible programmability, specifically designed to build the next generation of observability pipelines. Leveraging Logtail's robust log collection and processing capabilities, LoongCollector has been extensively enhanced with new features and extensions. Its functionality is evolving from a singular log-focused approach to a comprehensive system for observable data collection, local computing, and service discovery.

Core advantages

Rich data sources

Embracing an all-in-one design philosophy, LoongCollector aims to handle all collection tasks with a single agent. It manages the collection, processing, routing, and sending of logs, metrics, traces, events, and profiles. In the future, LoongCollector's Prometheus scraping capabilities will be enhanced, Extended Berkeley Packet Filter (eBPF) technology will be integrated for non-intrusive collection, and native metric data collection features will be offered, positioning it as a OneAgent solution.

Kubernetes (K8s) collection scenarios represent a fundamental capability of LoongCollector. In the observability domain, K8s metadata, such as namespaces, pods, containers, and labels, is vital for analyzing observable data. LoongCollector utilizes the standard Container Runtime Interface (CRI) API and pod definitions to retrieve various K8s metadata, enabling automatic tagging during the collection process.

Flexible programming pipeline

LoongCollector features a versatile programmable system supported by Simple Log Service Processing Language (SPL) and multi-language plugin dual engines. These engines interconnect to deliver the desired computing power through various combinations.

Depending on their requirements, users can choose between native plugins for execution efficiency and SPL engines for a comprehensive set of operators and complex data processing.

Programmable engine	Classification	Characteristics
Multi-language plugin engine	Native plugin	Implemented in C++ High performance, very low resource overhead Relatively complete operator capability
Multi-language plugin engine	Extension plugin	Implemented in Golang Relatively high performance, low resource overhead Relatively complete operator capability
SPL engine	SPL engine	Implemented in C++ Columnar model, vectorized execution High performance, low resource overhead Comprehensive operator capability Pipeline design, flexible combination, can handle complex data

High performance and high reliability

LoongCollector consistently strives for high collection performance and reliability. Its core value is to provide a stable and efficient unified observable data collection agent and end-to-end pipeline for large-scale distributed systems. Through ongoing technological innovation and optimization, LoongCollector's resource utilization is continually enhanced, allowing it to maintain stable operations in demanding environments.