Write a Wasm plug-in in Go for an Envoy proxy - Alibaba Cloud Service Mesh

WebAssembly for Proxies is a new specification that allows developers to write portable plug-ins using WebAssembly (Wasm). These plug-ins can run on various proxy servers. Service Mesh (ASM) supports the WebAssembly for Proxies specification. This topic describes how to write a Wasm plug-in in Go for an Envoy proxy in ASM.

Prerequisites

A cluster is added to an ASM instance whose version is 1.18 or later. For more information about how to add a cluster to an ASM instance, see Add a cluster to an ASM instance.
Automatic sidecar proxy injection is enabled. For more information, see Configure sidecar proxy injection policies.
An ingress gateway is deployed. For more information, see Create an ingress gateway.
An HTTPBin application is deployed and can be accessed. For more information about how to deploy an HTTPBin application, see Deploy the HTTPBin application.
A Container Registry Enterprise Edition instance is created. Container Registry Enterprise Edition instances support Open Container Initiative (OCI) images. For more information, see Create a Container Registry Enterprise Edition instance.

Background information

Wasm is an up-and-coming and portable binary format for executable code. The code is executed at a nearly-native speed in a memory-safe (for host) sandbox. The sandbox has clearly defined resource constraints and provides a clearly defined API for communicating with the embedding host environment (here refers to a proxy).

Wasm plug-ins provide the following benefits:

Agility: You can update plug-ins without restarting the Envoy proxy. Therefore, requests can be handled as expected.
Reliability and isolation: Because plug-ins are deployed inside a sandbox with resource constraints, they can crash without bringing the Envoy proxy down.
Security: Because plug-ins are deployed inside a sandbox with a clearly defined API for communicating with a proxy, they are well controlled.
Diversity: Plug-ins can be written in multiple programming languages, such as C++, Go, and Rust.

For more information about Wasm plug-ins, see WebAssembly-in-Envoy.md and OVERVIEW.md.

Configuration example

In this example, a Wasm plug-in is written in Go. After the plug-in is written, a Wasm binary file is generated, and then packaged into an image. The image must be uploaded to an OCI image repository. After the image is uploaded, configure the WasmPlugin resource in ASM and apply the plug-in to the specified Envoy proxy.

In this example, a plug-in is developed to check whether a request contains the allow: true header. If no, the status code 403 and the specified body are returned. If yes, the HTTPBin application can be accessed as expected.

Step 1: Prepare a development environment

To develop a Wasm plug-in in Go for an Envoy proxy, you must install the following tools first:

Go: The Go compiler and related tools are used to write Go projects. For more information, see The Go Programming Language.
Docker: In this example, Docker is used to build and push OCI images.
TinyGo: Go is used to write the Wasm plug-in. However, you cannot use the official Go compiler to compile Go code into Wasm format. You must use TinyGo. For more information about how to install TinyGo, see Quick install guide.

For more information about the SDKs on which the Wasm plug-in depends, see proxy-wasm-go-sdk on the GitHub website. You can find the complete code of the Go SDK for Proxy-Wasm on the website. If you want to use other SDKs, refer to the code of the Go SDK for Proxy-Wasm.

Step 2: Write plug-in code

Create a folder and create a main.go file that contains the following content:

Show the main.go file

package main

import (
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
)

func main() {
	proxywasm.SetVMContext(&vmContext{})
}

type vmContext struct {
	// Embed the default VM context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultVMContext
}

// Override types.DefaultVMContext.
func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
	return &pluginContext{}
}

type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultPluginContext
}

// Override types.DefaultPluginContext.
func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
	return types.OnPluginStartStatusOK
}

// Override types.DefaultPluginContext.
func (ctx *pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
	return &HeaderAuthorizationHandler{}
}

type HeaderAuthorizationHandler struct {
	// Embed the default http context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultHttpContext
}

// Override types.DefaultHttpContext.
func (ctx *HeaderAuthorizationHandler) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
	// Randomly routing to the canary cluster.

	const AuthorizationKey = "allow"
	value, err := proxywasm.GetHttpRequestHeader(AuthorizationKey)
	if err != nil || value != "true" {
		proxywasm.LogDebugf("request header: 'allow' is %v, only true can passthrough", value)
		return ctx.DenyRequest()
	}
	return types.ActionContinue
}

func (ctx *HeaderAuthorizationHandler) DenyRequest() types.Action {
	proxywasm.SendHttpResponse(403, [][2]string{{"Content-Type", "text/plain"}}, []byte("Forbidden by ASM Wasm Plugin"), -1)
	return types.ActionPause
}

Run the following commands in the created folder to obtain the SDK dependencies:
```
go mod init
go mod tidy
```
Run the following command to compile the code into a Wasm binary file:
```
tinygo build -o plugin.wasm -scheduler=none -target=wasi main.go
```
A plugin.wasm file is generated. This file is the Wasm binary executable file.

Step 3: Create an OCI image of the Wasm plug-in and push it to the Container Registry Enterprise Edition instance

In the folder created in Step 2, create a Dockerfile file that contains the following content:
```
FROM scratch
ADD ./plugin.wasm ./plugin.wasm
```
Run the following commands to create an image:
```
docker build -t header-authorization:v0.0.1 .
```
Creates an image repository. For more information, see substeps 2.a and 2.b of Step 1 in the Use the Coraza Wasm plug-in to implement WAF capabilities on an ASM gateway topic.
In this example, the namespace is test-oci and the repository name is header-authorization. The following figure shows the created repository.
For more information about how to push an image to the Container Registry Enterprise Edition instance, see Push image to the registry in the preceding figure.

Step 4: Apply the Wasm plug-in to the ingress gateway

Configure permissions to pull images. For more information, see Step 2: Configure permissions to pull images.

Run the following command to create a Secret named wasm-secret:

kubectl create secret docker-registry -n istio-system wasm-secret --docker-server=${Domain name of the Container Registry Enterprise Edition instance} --docker-username =${Username} --docker-password =${Password}

Create an asm-plugin.yaml file that contains the following content:

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: header-authorization
  namespace: istio-system
spec:
  imagePullPolicy: IfNotPresent
  imagePullSecret: wasm-secret
  selector:
    matchLabels:
      istio: ingressgateway
  url: oci://${Domain name of the Container Registry Enterprise Edition instance}/test-oci/header-authorization:v0.0.1
  phase: AUTHN

Use kubectl to connect to the ASM instance based on the information in the kubeconfig file. Then, run the following command to apply the Wasm plug-in to the ASM instance:
```
kubectl apply -f wasm-plugin.yaml
```

Step 5: Verify that the Wasm plug-in takes effect

Use the kubeconfig file of the cluster on the data plane in which the ingress gateway resides and run the following command to enable the debug logging feature for the Wasm plug-in that is applied to the ingress gateway:
```
kubectl -n istio-system exec ${Name of the pod where the ingress gateway resides} -c istio-proxy -- curl -XPOST "localhost:15000/logging?wasm=debug"
```
Run the following command to access the HTTPBin application exposed over the ingress gateway:
```
curl ${IP address of the ingress gateway}/status/418
```
Expected output:
```
Forbidden by ASM Wasm Plugin
```

View the logs of the pod where the ingress gateway resides.

Sample logs:

2024-03-08T08:16:46.747394Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1168	wasm log istio-system.header-authorization: request header: 'allow' is , only true can passthrough	thread=24
{"bytes_received":"0","bytes_sent":"28","downstream_local_address":"xxxxxxx","downstream_remote_address":"xxxxxxxx","duration":"0","istio_policy_status":"-","method":"GET","path":"/status/418","protocol":"HTTP/1.1","request_id":"780c8493-13e4-4f97-9771-486efe30347c","requested_server_name":"-","response_code":"403","response_flags":"-","route_name":"httpbin","start_time":"2024-03-08T08:16:46.747Z","trace_id":"-","upstream_cluster":"outbound|8000||httpbin.default.svc.cluster.local","upstream_host":"-","upstream_local_address":"-","upstream_service_time":"-","upstream_response_time":"-","upstream_transport_failure_reason":"-","user_agent":"curl/8.4.0","x_forwarded_for":"xxxxxx","authority_for":"xxxxxx"}

Run the following command to access the HTTPBin application exposed over the ingress gateway:

curl ${IP address of the ingress gateway}/status/418 -H "allow: true"

Expected output:

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`

The output indicates that the HTTPBin application can be accessed as expected.

Memory leaks in TinyGo

Memory leaks exist in the Wasm plug-in for an Envoy proxy compiled by using TinyGo. The proxy-wasm-go-sdk community recommends that you use nottinygc for compilation optimization. Perform the following steps to use nottinygc for compilation optimization:

Add the following import code to the beginning of the main.go file:
```
import _ "github.com/wasilibs/nottinygc"
```
If no dependencies are found, you can run the go mod tidy command to automatically download the dependencies.
Run the following command to compile the code:
```
tinygo build -o plugin.wasm -gc=custom -tags='custommalloc nottinygc_envoy'  -target=wasi -scheduler=none main.go
```
The preceding command sets the -gc and -tags parameters. For more information, see nottinygc.