eunomia-bpf: The Lightweight Development Framework for eBPF and WebAssembly Is Now Available!

Recently, at the Apsara Conference 2022 OpenAnolis Forum-eBPF & Linux Stability Session, Zhejiang University’s Yusheng Zheng (a maintainer of the eBPF Technology Exploration SIG) introduced eunomia-bpf: Lightweight Development Framework for eBPF and Wasm. The following article highlights the main points of that speech.

Hello everyone! I am Yusheng Zheng from Zhejiang University. Today, I would like to introduce the eunomia-bpf project. It is a lightweight eBPF development framework designed to simplify the development, distribution, and operation of eBPF programs, and running eBPF programs in multi languages and Webassembly.

Through some simple examples, I will show how eunomia-bpf can download and run eBPF programs from the cloud with the one-line command, only write kernel code to run and export events, and the combination with WebAssembly (Wasm) and other functions. Finally, I will briefly explain the principles of eunomia-bpf, its design ideas and implementation, and the next development direction.

1. Overview

Eunomia-bpf originated from an idea during the 2022 National College Student Operating System Competition. It aims to run the eBPF program as a service, package the eBPF program as a JSON object, and dynamically plug and unplug any relocatable eBPF program through HTTP requests. It is expected to adapt to different kernel versions and architectures. After the competition, with the help and guidance of several university teachers and some community partners, these ideas were gradually turned into a rudimentary open-source project. (Here, I would like to thank Professor Lijun Chen of Xi’an University of Posts and Telecommunications and the team and Wenan Mao from OpenAnolis.) Currently, the team includes me, the team of Professor Lijun Chen, and friends from OpenAnolis.

eunomia-bpf aims to solve the following two problems (or the main pain points in the development and distribution of eBPF programs):

The threshold for building and developing eBPF programs is high for novices. It is necessary to pay attention to the interaction and information processing of both the kernel mode and the user mode and write complex or repetitive user mode loading code.
It is impossible to easily and quickly package, distribute, and release various eBPF programs on different kernel versions of different architectures. Many eBPF gadgets are developed in different languages and have different interfaces, which cannot be easily integrated into large observable systems. Currently, there is no good plug-in solution. The entire observable framework must be recompiled and redeployed for a large eBPF application to launch before the eBPF probe or data processing module can be updated. In addition, if uncensored and untested third-party user-mode data processing code is introduced, a code crash may also cause the entire program to crash.

Therefore, we propose three solutions:

For beginners, you only need to write kernel mode code to automatically sample and automatically obtain the data exported by the kernel mode. After compilation, you can distribute, load, and run the program. This significantly reduces the learning cost of eBPF and improves development efficiency.
Second, based on the libbpf feature of Compile Once, Run Everywhere, separate the compilation and running of user mode and kernel mode completely and distribute the program through standard JSON or Wasm modules without recompilation. Application startup consumes fewer resources and takes less time, and container startup is shorter.

Wasm is a binary instruction format for a stack-based virtual machine. It is designed for portability. It can be used as a compilation target for high-level languages (such as C/C+/RUST), enabling client and server applications to be deployed on the Web. Wasm has developed into a lightweight, high-performance, cross-platform, and multilingual software sandbox environment, which is used in cloud-native software components and can run in non-browser environments. The design idea of Wasm also has many similarities with eBPF.

When you only write kernel mode code, JSON can be used to complete the process of distribution, loading, and packaging. For complete eBPF applications or tools that require interaction between the user mode and the kernel mode, you can write complex user mode processing programs in Wasm, with multiple languages to control and process the data collected by eBPF programs, embed pre-compiled eBPF bytecode in the Wasm module for distribution, dynamically load and run it on the target machine. Currently the eunomia-bpf project has provided C/C++, Rust SDKs for developing eBPF programs in Wasm, and more language supports(Such as Go in Wasm) will be added later.

The combination with Wasm ecological items can bring many features to eBPF programs. At the same time, it coincides with the original design ideas of eBPF programs (such as portability, isolation, and security), and it is a cross-language and lightweight running environment. Also, the storage and distribution of OCI images of eBPF programs can be completed with the help of Wasm-related tools. Recently, Docker officials launched a Wasm-based distribution and operation tool.

The three parts above are the core features of eunomia-bpf. Let's look at some examples.

2. Examples

Eunomia-bpf is not a complete system but development libraries, compilers and tools that can be easily embedded in a large toolchain (like coolbpf) or embedded as a runtime library anywhere we need to use eBPF as a plug-in.

The precompiled eBPF program can be downloaded directly from the web page url with the one-line command. Use the WebAssembly module or JSON configuration file for distribution the eBPF programs without recompiling during deployment. The startup speed is one to two orders of magnitude faster than the BCC.

In the figure above, the URL used in starting the eBPF program can be replaced by OCI images or Docker images, which can be stored in Docker Repository or GitHub Package. The usage method is virtually the same as Docker. You only need to execute pull and run, or you can directly push the compiled package to use. Compared with traditional Docker images, Wasm, as a lightweight container, has a faster startup speed and retains the important feature of eBPF. It can be easily embedded into other programs as a submodule or plug-in and is completely independent of the specific kernel version and architecture.

With eunomia-bpf, you only need to write kernel mode code to run the program correctly, minimizing the barriers for newcomers to get started. Without writing libbpf user mode loading framework, you can automatically export kernel mode perf event or ring buffer event, for example:

Also, you can automatically collect the data in the hash map and generate a histogram, as shown in the following code:

The corresponding user mode loading code can be automatically generated, and the information in the corresponding hist map can be printed as a histogram by writing the corresponding annotation information in the kernel mode:

You only need to add an annotation to the corresponding global variable position in the code of the kernel mode to automatically generate the command line parameters of the corresponding tool, for example:

The command-line help information is listed below:

In addition, it is fully compatible with native libbpf. After obtaining the kernel mode code of libbpf tools, it can be packaged, released, and run directly without modifying any code.

You can add tracepoints or other content in the form of annotation. You can use a container to package and compile a toolchain without worrying about environment configuration issues. The one-line command generates a project template, and the one-line command does compilation.

Generally speaking, the complete eBPF application is divided into two parts: user-space program and kernel program. User-space program is responsible for loading BPF bytecode to the kernel or reading statistics or event details returned by the kernel to carry out relevant data processing and control.

We can write the user-mode auxiliary program in Wasm to complete safe and efficient user-mode data processing and control logic. It also has the features of eBPF, such as security, portability, lightweight, modularity, etc. (Wasm is also a sandbox environment like eBPF. Even if the Wasm module crashes when it is running in the user mode, it will not cause the crash of the host program.) It can also be used as a plug-in. When adding new data processing logic, you do not need to change the original code. (Note: Wasm is optional rather than necessary. Writing kernel mode code is sufficient for some simple applications.)

We write the code in the C language and package it to generate the Wasm module. After that, we can:

Distribute and manage eBPF programs (such as docker-Wasm) with the help of WebAssembly-related ecological items
Embed them in large applications as eBPF programmable modules or plug-ins

Here is a simple Wasm module that can obtain the signal transmission events between processes of the current system. It can accept some command line parameters and process the reported information.

Currently, we can compile the programs in bcc/libbpf-tools into Wasm modules without code modification. In terms of development experience, it can also be the same as the eBPF program that uses the C/C++/Rust language to develop libbpf. Later, you can also try to introduce other languages to develop SDK (such as Go, Java, etc.).

The main difficulty in combining Wasm and eBPF is that the memory layout of Wasm is different from the eBPF program, and the struct of the C language cannot be mapped directly, so we create some specify toolchain for generating eBPF-Wasm language bindings. At the same time, Wasm has many restrictions on accessing system resources (such as files and networks). Many standard libraries are missing, so we need to carry out some special processing and transplantation in Wasm modules.

3. System Architecture

The underlying architecture depends on the infrastructure of the kernel mode and the user mode (such as the libbpf library and kernel), providing relevant compilation toolchains to help generate eBPF JSON skeleton or package code into Wasm modules. The toolchains include Clang, LLVM, and bpftool. The dynamic loading library can be used independently and has nothing to do with Wasm. It can dynamically load eBPF programs based on JSON information and can easily implement kernel functions as your service through HTTP interfaces (the form of kernel functions and services).

We have also implemented the Wasm abstraction layer, which contains API specifications (such as the access form occupied by the WSAI system of Wasm) or the access form interacted with eBPF. There are also Wasm-based customized libbpf libraries, ported auxiliary state programs, and serialization libraries, which are used to load libbpf-based eBPF programs in Wasm modules.

Runtime libraries can be easily replaced (such as the Wasm runtime of WSI). In addition, the upper layer implements LMP, command line tools, and observability tools.

The eunomia-bpf project is open-source in OpenAnolis. You are welcome to try it and give suggestions and feedback to improve it.

Please see the following links for more information:

Address link of the eunomia-bpf project: https://github.com/eunomia-bpf/eunomia-bpf
Tools for generating C structs serialization-free bindings between eBPF and Wasm: https://github.com/eunomia-bpf/c-struct-bindgen
examples of eunomia-bpf framework: https://github.com/eunomia-bpf/eunomia-bpf/tree/master/examples
Address link of the libbpf-Wasm library: https://github.com/eunomia-bpf/libbpf-Wasm
Address link of eBPF Technology Exploration SIG: https://openanolis.cn/sig/ebpf-research

Community

eunomia-bpf: The Lightweight Development Framework for eBPF and WebAssembly Is Now Available!

1. Overview

2. Examples

3. System Architecture

Read previous post:

Read next post:

OpenAnolis

You may also like

Comments

OpenAnolis

Related Products

YiDA Low-code Development Platform

mPaaS

Managed Service for Prometheus

Apsara Stack