Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Higress's New WASM Runtime Greatly Improves Performance

Higress's New WASM Runtime Greatly Improves Performance

This article introduces the latest progress of Higress in switching the runtime of Wasm plug-in from V8 to WAMR.

This article introduces the latest progress of Higress in switching the runtime of Wasm plug-in from V8 to WebAssembly Micro Runtime (WAMR). By switching to WAMR and turning on the AOT mode, the Wasm plug-in performance has been greatly improved by 50% on average, and the performance of some plug-ins with complex logic has been doubled.

The Higress and Wasm plugin

Higress is a cloud-native api gateway open-source project, powered by Istio and Envoy, and largely adopted by Alibaba Cloud.

Alibaba Cloud has offered online service of Wasm plug-in development and deployment for their Higress product since 2022, which has grown to the main gateway expansion method because of the unique value to users:

  1. Engineering reliability: Compared with dynamic type + interpreted execution languages such as Lua, Wasm can be compiled based on a variety of static type languages and can be checked during compilation to avoid runtime errors and turn the production environment into a code debugging scene.
  2. Sandbox security: The Wasm plug-in runs in a strict virtual machine sandbox environment and has its own independent memory space. It cannot directly access the external memory, which can avoid plug-in code bugs that lead to buffer overflow, remote code execution, and other attacks.
  3. Hot update: Higress is based on Envoy's xDS mechanism. Both plug-in binaries and configurations can be hot updated independently without causing connection disconnection. It is more friendly to business scenarios with long-lived connections, such as WebSocket and gRPC.
  4. Strong performance: Higress's Wasm plug-in has 3 times the performance improvement compared to the traditional Spring Cloud Gateway plug-in architecture.

Based on the Istio/Envoy, Higress adds three core capabilities to the Wasm plug-in mechanism:

  1. Domain name/routing level validation: The global validation method provided by Istio/Envoy is difficult to meet the needs of most scenarios. The plug-in developed based on Higress Wasm SDK can achieve this, and the compiled plug-in is also compatible with the Istio/Envoy ecosystem, (only takes effect globally).
  2. Redis access capability: Higress provides serval host functions to access Redis. Plug-in code can implement multiple capabilities based on Redis, such as global flow restriction, session state management, etc.
  3. Virtual machine self-healing mechanism: If problems such as null pointer access, array out-of-bounds, memory leaks, etc. occur in the plug-in logic, they will be captured by the runtime and will not cause the gateway to crash. Higress supports automatic restart after the Wasm module fails, and can quickly stop the bleeding. Users can get the callstack showing the cause of the problem via alerts.

From the perspective of Higress's enterprise users, the adoption life cycle of Wasm plug-in technology has crossed the chasm and entered the early majority stage. The core driving force is the cost reduction brought by performance dividends. For authentication, encryption, decryption, session management, and other logic, computing resources are offloaded at the gateway without the need for back-end service processing, thereby reducing computing costs globally.

Wasm Runtime Upgrade: from V8 to WAMR

Problems with V8

Wasm technology was born in the browser scene and V8 is the JS/WebAssembly engine of Chromium. The V8 engine with JIT mode has good performance in executing the Wasm modules. But there are also the following problems with the practices:

1.  Complexity: The V8 project is very complex, and Wasm-related implementations are highly coupled with JS processing logic. For example, a bug in the early Envoy Wasm plug-in was caused by V8 introducing pointer compression to optimize JS execution memory.

Bugs:
https://bugs.chromium.org/p/v8/issues/detail?id=12592

2.  Lack of collaboration between the V8 community and the Envoy community. Envoy’s current version dependence on V8 is still stuck in 2022, and it cannot support new features such as Wasm GC. Because the project is complex, the risk of upgrading V8 dependencies is also quite high.

3.  Client-side orientation: Most V8 users and developers come from the client-side. Considering device compatibility, they prefer JIT mode. AOT mode does not have significant performance improvements and cannot fully utilize the performance advantages of Wasm.

Why Do We Choose WAMR?

WAMR is a popular WebAssembly runtime open-source project first developed by the Intel team and under the Bytecode Alliance (a non-profit organization for the Wasm software ecosystem). Currently, active contributors in the community include engineers from Intel, Xiaomi, Amazon, Sony, Midokura, Siemens, Ant Group, and other companies. WAMR is developed using C language and has excellent platform adaptability. It supports interpretation mode, just-in-time compilation, and ahead-of-time compilation modes to run Wasm modules. It has excellent performance and has performed well in multiple public performance evaluation reports. It also has extremely low resource overhead and can run a single Wasm instance in 100KB memory.

Performance Data and Improvement

• Pressure measurement tool: k6
• Server CPU model: Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz
• Stress test method: Higress starts 2 worker threads, fixes the pressure of k6 during the stress test, and runs both threads

Some Higress plug-ins were selected for performance testing. The situation is as follows:

table

Note: The data in the table is the average additional delay of a single request.

Overall, the more complex the Wasm command of the plug-in, the more obvious the WAMR improvement. All the above plug-ins, except jwt-logout, which is an enterprise version plug-in and is not open-source, the other plug-ins can be viewed in the Higress open-source warehouse directory for corresponding source code implementation: https://github.com/alibaba/higress/tree/main/plugins/wasm-cpp/extensions

To compile and generate AOT files, you can use wamrc, the official compilation tool provided by WAMR: wamrc --invoke-c-api-import -o plugin.aot plugin.wasm

In order for the generated wasm file to be compatible with JIT mode, use the script under the WAMR warehouse to generate the merged file: python3 wasm-micro-runtime/test-tools/append-aot-to-wasm/append_aot_to_wasm.py --aot plugin.aot - -wasm plugin.wasm -o plugin.aot.wasm

Taking the oauth plug-in with the greatest improvement as an example, you can use the following configuration to reproduce:

k6 stress test command:

k6 run --vus 300 ./script.js --duration 60s

k6 stress test script:

import http from 'k6/http';
import { check } from 'k6';

export default function () {
    const res = http.get('http://11.164.3.16:10000/',{headers: {'Authorization':'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6ImFwcGxpY2F0aW9uL2F0K2p3dCJ9.eyJhdWQiOiJ0ZXN0MiIsImNsaWVudF9pZCI6Ijk1MTViNTY0LTBiMWQtMTFlZS05YzRjLTAwMTYzZTEyNTBiNSIsImV4cCI6MTY2NTY3MzgyOSwiaWF0IjoxNjY1NjczODE5LCJpc3MiOiJIaWdyZXNzLUdhdGV3YXkiLCJqdGkiOiIxMDk1OWQxYi04ZDYxLTRkZWMtYmVhNy05NDgxMDM3NWI2M2MiLCJzY29wZSI6InRlc3QiLCJzdWIiOiJjb25zdW1lcjEifQ.LsZ6mlRxlaqWa0IAZgmGVuDgypRbctkTcOyoCxqLrHY'}});
    check(res, { 'status was 200': (r) => r.status == 200 });
}

envoy configuration snippet


                  - name: envoy.filters.http.wasm
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      config:
                        name: "my_plugin"
                        configuration:
                          "@type": "type.googleapis.com/google.protobuf.StringValue"
                          value: |
                            {
                                "consumers": [
                                    {
                                        "name": "consumer1",
                                        "client_id": "9515b564-0b1d-11ee-9c4c-00163e1250b5",
                                        "client_secret": "9e55de56-0b1d-11ee-b8ec-00163e1250b5"
                                    }
                                ],
                                "clock_skew_seconds": 3153600000
                            }
                        vm_config:
                          runtime: envoy.wasm.runtime.wamr
                          #runtime: envoy.wasm.runtime.v8
                          code:
                            local:
                             filename: "oauth.aot.wasm"
                          allow_precompiled: true

Reasons for Performance Improvement

The main reasons include:

  1. WAMR provides deeply optimized pre-compilation capabilities. Before deployment, WAMR translates Wasm opcodes into IR, and generates machine code for the specified platform through a customized optimization pipeline. At runtime, executing pre-compiled Wasm can achieve performance comparable to a native binary.
  2. WAMR uses a highly optimized FFI. Effectively reduce the number of type conversions and memory copies required when shuttle between the host (C/C++) and guest (Wasm) worlds, reducing unnecessary losses.
  3. WAMR can intelligently sense the hardware acceleration capabilities of the platform and make full use of them. For example, when running on the x86 platform, WAMR implements the segue algorithm recently proposed by the academic community, using the GS register as the addressing method to improve the efficiency of accessing WASM linear space.

Future Outlook

With the close collaboration between the Higress community and the WAMR community, in addition to improving the Wasm plug-in performance in gateway scenarios, many practical new features will be released soon, so stay tuned:

1.  Supports generating CPU flame graph. For example, the following is the CPU flame graph seen when executing Fibonacci recursion in a Wasm plug-in:

1

2.  After a logic problem in Wasm plug-in causes a crash, the complete callstack can be printed in the plug-in log, and the specific line number in the source code can be located through the addr2line tool provided by WAMR.

3.  Supports observing the CPU and memory usage of each Wasm plug-in module.

4.  Supports using TypeScript to write Wasm plug-ins, complete syntax support.

More developers are welcome to participate in the Higress and WAMR open-source communities.

GitHub project addresses:

About the Authors

  • Tianyi Zhang: Alibaba Cloud API Gateway developer, Higress community maintainer
  • Liang He: Intel Web Platform Engineering developer, WAMR community TSC member
performance Computing Open Source Istio WebAssembly WASM Higress WebAssembly Micro Runtime
0 1 0
Share on

Read previous post:

Decipher the Knative's Open-Source Serverless Framework: Traffic Perspective

Read next post:

Application Monitoring eBPF Edition: Non-intrusive Application Monitoring of the Golang Microservice

Alibaba Cloud Native Community

508 posts | 49 followers

You may also like

Comments

Alibaba Cloud Native Community

508 posts | 49 followers

Related Products

More Posts by Alibaba Cloud Native Community

See All