By OpenAnolis System Operation & Maintenance Alliance(SOMA)
SOMA primarily focuses on collaborating with institutes like China Academy of Information and Communications Technology (CAICT), universities, platforms, and O&M manufacturers to establish evaluation standards and systems for OS business systems and O&M systems. Coolbpf serves as the basic component for data collection of SOMA's evaluation system and is also the eBPF data collection platform on the node end of the SysOM operations system. In response to SOMA's work and business development needs, Coolbpf has undergone numerous updates recently, introducing multiple open-source features and newly developed features. Specifically, it now supports the eNetSTL network functionality, user-space probes (bpftime that reduces overhead by not requiring switches between user space and kernel space), eBPF packet capture (rdump), and multi-language continuous profiling.
With the popularization and extensive application of eBPF technology in production environments, the setup of compilation environments has become quite mature. Additionally, as lower kernel versions such as 3.10 are gradually migrating to versions 4.19 and even 5.10, Coolbpf plans to remove remote compilation and low-version driver features. In the future, Coolbpf will focus more on providing specific functionalities, such as user-space probes, software networking, and profiling. These features will be offered in the form of libraries to specific applications at a higher level, allowing developers to focus less on the granular metrics collection at the system level and more on their own business application development. They can directly call Coolbpf library functions to satisfy their requirements.
The following figure shows the features of Coolbpf:
However, the highly frequented BTF download service will not be discontinued. This ensures that users can utilize eBPF on kernel versions 4.19 and some 5.10 that do not default to publishing BTF files, without needing to generate the BTF files themselves. Since its release, the BTF download website has been called more than 200 million times.
BTF download website: https://mirrors.openanolis.cn/coolbpf/btf/
In response to the recent updates of Coolbpf, we will introduce each feature and usage through a series of articles. First of all, let's look at the eNetSTL software network function acceleration library, contributed by the team of Professor Dian Shen from Southeast University (authors: Dian Shen, Bin Yang, Hanlin Yang, and Lunqi Zhao). The code has been merged into https://gitee.com/anolis/coolbpf
eNetSTL first uses kfunc technology to provide a software network function library, including switch and router functionalities, which has attracted widespread attention in both the industry and academia. For code structures that require more access to kernel data, such as virtio-net and I/O queue polling tools, you can also refer to the implementation of eNetSTL to write functions based on kfunc.
eNetSTL is an eBPF network function acceleration library based on eBPF's kfunc and kptr technologies. It accelerates eBPF-based network functions without modifying the eBPF infrastructure (such as the instruction sets) by abstracting certain common functions that are either impossible to implement or fail to meet performance standards into eNetSTL.
eNetSTL abstracts the aforementioned common patterns and implements them as a series of high-performance, low-overhead APIs. While solving problems, it avoids excessive code expansion. eNetSTL is implemented based on eBPF's kernel function (kfunc) and kernel pointer (kptr) technologies. It implements its APIs within kernel modules, which avoids the need for kernel modifications. Currently, eNetSTL is designed to be self-contained except for the use of kfunc and kptr interfaces, so it can maintain good compatibility across various kernel versions.
The contents included in eNetSTL are shown in the figure below:
Specifically, eNetSTL includes the following contents:
1. Memory wrapper: Support the use of non-contiguous memory in eBPF without breaking the security guarantees provided by eBPF.
2. Algorithms: Include bitwise operations, SIMD-based parallel hash computing, and parallel comparison algorithms.
3. Data structure: List bucket data structure, supporting a random number pool with GEO (Geometric Random Number) distribution.
The implementation of the memory wrapper makes full use of the kfunc and kptr technologies. Its main design includes:
The following is part of the API for the memory wrapper:
Currently, eNetSTL in Coolbpf includes a set of algorithms that enhance the performance of eBPF-based network functions, including bitwise operations, SIMD-based parallel hash computing, and parallel comparison algorithms. Other components are undergoing engineering adaptation, and the eNetSTL features in Coolbpf will be continuously updated in the future.
The following tools are required to compile the eNetSTL driver:
• Install the GCC compiler
• Install the kernel-devel and kernel-headers
• Install pahole (used to generate BTF information)
After installing the above compilation environment, use the command cd bpf_kernel_modules/eNetSTL && make to compile the driver. The driver file generated after compilation is bpf_kernel_modules/eNetSTL/eNetSTL.ko. Install the driver by using the command insmod eNetSTL.ko.
The following tools are required to compile the eNetSTL sample program:
• Install Clang/LLVM
• Install the Rust compilation toolchain, as Coolbpf requires a Rust environment.
After installing the above compilation environment, run the command mkdir build && cd build to create a compilation directory, and then run the command cmake -DBUILD_EXAMPLE=on .. to enable the compilation of the sample program and complete the configuration before compilation. Next, run make to execute the compilation. The compiled eNetSTL program will be located in: build/tools/examples/eNetSTL/enetstl_countmin_sketch and build/tools/examples/eNetSTL/enetstl_cuckoo_hash.
enetstl_countmin_sketch
The enetstl_countmin_sketch tool utilizes the eNetSTL kernel driver, which implements the Count-Min Sketch algorithm through the eBPF kfunc. The Count-Min Sketch algorithm is a probabilistic data structure, which is mainly used to deal with frequency estimation in large-scale data streams. The following is an example of the runtime output of the tool, showing that it has successfully passed the test cases, which indicates that the eNetSTL function is operating normally.
# build/tools/examples/eNetSTL/enetstl_countmin_sketch
test:PASS:bpf_prog_test_run_opts res 0 nsec
test:PASS:enetstl_countmin_sketch_bpf:test_countmin 0 nsecenetstl_cuckoo_hash
The enetstl_cuckoo_hash tool utilizes the eNetSTL kernel driver, which implements the cuckoo hash algorithm through the eBPF kfunc. The cuckoo hash algorithm is a hash table technology that uses two hash functions and a dynamic conflict resolution mechanism to provide fast lookup and deletion operations. The following is an example of the runtime output of the tool, showing that it has successfully passed the test cases, which indicates that the eNetSTL function is operating normally.
# build/tools/examples/eNetSTL/enetstl_cuckoo_hash
test:PASS:bpf_prog_test_run_opts res 0 nsec
test:PASS:enetstl_cuckoo_hash_bpf:test_cuckoo_hash 0 nsecThe above content introduces some recent feature updates in Coolbpf and introduces eNetSTL functions and usage in detail. For more information, please refer to the Coolbpf project source code.
AI Software Stack-Oriented Optimized Design - AI Ecosystem Construction in OpenAnolis
87 posts | 5 followers
FollowOpenAnolis - November 29, 2022
OpenAnolis - July 27, 2023
OpenAnolis - February 8, 2023
OpenAnolis - April 7, 2023
OpenAnolis - February 2, 2023
OpenAnolis - December 24, 2024
87 posts | 5 followers
Follow
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn MoreMore Posts by OpenAnolis