Alibaba Cloud Service Mesh FAQ (1): How to Use the WebSocket over HTTP/2 Protocol

By Xining Wang

Alibaba Cloud released the commercial version of ASM on April 1, 2022, to meet the increasing needs of enterprises for large-scale use of Alibaba Cloud Service Mesh (ASM) products, multilingual service interoperability, and fine-grained service governance. It aims to provide enterprises with guarantees in performance, security, high availability, and high reliability for large-scale implementation of service mesh capabilities in production environments.

Alibaba Cloud has been researching and practicing Service Mesh for a long time. Alibaba Cloud has continuously driven the development of Service Mesh technology and accumulated a series of core technologies by summarizing the experience in business scenarios. ASM has become the industry's first managed Service Mesh platform compatible with Istio to empower users on the cloud. This series of articles will summarize common problems and solutions when using Service Mesh technology.

Background

WebSocket is a protocol for two-way communication between the client and the server. It is based on the RFC 6455 standard and is widely used by applications and API implementations. WebSocket is different from HTTP, as it uses the HTTP Upgrade header to establish the connection between parties. Istio cannot recognize the WebSocket protocol. However, Istio Sidecar Proxy provides out-of-the-box support for WebSocket without additional configuration. In the following example, we will show a running WebSocket application and check the configuration related to the WebSocket protocol contained in the Envoy configuration.

Envoy's support for WebSocket is documented in the following links:

Istio protocol selection and discovery are described in the article below:

Protocol Selection

We will deploy a WebSocket application in our Kubernetes cluster using a Python library provided by the WebSocket community. The documentation for WebSocket contains a sample code and a yaml list that we can use to create a WebSocket application in a Kubernetes cluster. If you need to modify the deployment configuration in this example, follow the steps in the containerized applications section of this article to create a Python application and rebuild a Docker mirroring.

Then, modify the deployment configuration in the example to run the WebSocket server in the Istio mesh.

The following are the yaml files that can be used for deployment:

apiVersion: v1
kind: Service
metadata:
  name: websockets-server
  labels:
    app: websockets-server
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: 80
      name: http-websocket
  selector:
    app: websockets-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: websockets-server
  labels:
    app: websockets-server
spec:
  selector:
    matchLabels:
      app: websockets-server
  template:
    metadata:
      labels:
        app: websockets-server
    spec:
      containers:
      - name: websockets-test
        image: registry.cn-beijing.aliyuncs.com/aliacs-app-catalog/istio-websockets-test:1.0
        ports:
        - containerPort: 80

Next, you need to deploy the application to a namespace where automatic Sidecar Proxy injection has been enabled in the Istio mesh.

Now, deploy the client-side WebSocket application, which you can use to open a WebSocket interactive communication. We will use the same WebSocket Python library to test our application, but we need to build a mirror with the pre-installed library.

You can use the following simple Dockerfile to create the mirror we will use.

FROM python:3.9-alpine
RUN pip3 install websockets

Alternatively, use the Docker mirror we have built. The following yaml list can be used to deploy to our cluster.

apiVersion: v1
kind: Service
metadata:
  name: websockets-client
  labels:
    app: websockets-client
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: 80
      name: http-websockets-client
  selector:
    app: websockets-client
---    
apiVersion: apps/v1
kind: Deployment
metadata:
  name: websockets-client-sleep
  labels:
    app: websockets-client
spec:
  selector:
    matchLabels:
      app: websockets-client
  template:
    metadata:
      labels:
        app: websockets-client
    spec:
      containers:
      - name: websockets-client
        image: registry.cn-beijing.aliyuncs.com/aliacs-app-catalog/istio-websockets-client-test:1.0
        command: ["sleep", "14d"]

Deploy the preceding yaml file to the same namespace as the WebSocket server application.

We need to open a shell command to the client POD to test the application.

kubectl exec -it -n <namespace> websockets-client-sleep-<xxxx>  -c websockets-client -- sh

You can use the command in the following example (note that you need to replace with the actual namespace, such as the default) to open a WebSocket interactive connection of our server.

python3 -m websockets ws://websockets-server.<namespace>.svc.cluster.local:8080

The following results are obtained:

Connected to ws://websockets-server.default.svc.cluster.local:8080.
> hello
< hello
> world
< world
Connection closed: 1000 (OK).

Run under the HTTP/1.1 Protocol

First, we run the example under Istio 1.11. The following shows how WebSocket connections appear in istio-proxy access logs.

Sidecar Proxy logs on the client are lisyrf brloe:

{"method":"GET","upstream_local_address":"10.208.0.27:34200","upstream_transport_failure_reason":null,"x_forwarded_for":null,"duration":7035,"path":"/","istio_policy_status":null,"downstream_remote_address":"10.208.0.27:57510","bytes_received":35,"requested_server_name":null,"downstream_local_address":"192.168.106.4:8080","user_agent":"Python/3.9 websockets/10.2","upstream_cluster":"outbound|8080||websockets-server.default.svc.cluster.local","response_code":101,"upstream_host":"10.208.0.105:80","bytes_sent":23,"protocol":"HTTP/1.1","request_id":"dbcb8ac4-0fb0-4acf-94ef-24b31d704489","trace_id":null,"start_time":"2022-03-17T06:30:28.303Z","response_flags":"UC","route_name":"default","authority":"websockets-server.default.svc.cluster.local:8080","upstream_service_time":null}

Sidecar Proxy logs on the server are listed below:

{"duration":6030,"bytes_received":35,"start_time":"2022-03-17T06:30:28.308Z","istio_policy_status":null,"route_name":"default","user_agent":"Python/3.9 websockets/10.2","upstream_service_time":null,"request_id":"dbcb8ac4-0fb0-4acf-94ef-24b31d704489","downstream_local_address":"10.208.0.105:80","upstream_local_address":"127.0.0.6:53983","protocol":"HTTP/1.1","requested_server_name":"outbound_.8080_._.websockets-server.default.svc.cluster.local","response_flags":"UC","upstream_cluster":"inbound|80||","upstream_host":"10.208.0.105:80","x_forwarded_for":null,"trace_id":null,"upstream_transport_failure_reason":null,"authority":"websockets-server.default.svc.cluster.local:8080","path":"/","method":"GET","downstream_remote_address":"10.208.0.27:34200","bytes_sent":23,"response_code":101}

Even if we send multiple requests to the server, there is only one access log entry for the WebSocket connection. Envoy will only create this access log entry after the connection is closed, and we will not see any messages for each tcp request. This is because Envoy treats the WebSocket connection as the TCP byte stream, and the proxy can only understand the HTTP upgrade request/response.

HTTP2 Protocol Upgrade Issues

For Istio versions earlier than 1.12, something may go wrong in the HTTP2 upgrade process of WebSocket (which means it cannot connect to the WebSocket server normally but return a 503 error). The 503 results are returned (as shown in the figure):

$ python3 -m websockets ws://websockets-server.default.svc.cluster.local:8080
Failed to connect to ws://websockets-server.default.svc.cluster.local:8080: server rejected WebSocket connection: HTTP 503.

In other words, in specified traffic methods, h2UpgradePolicy cannot be set to UPGRADE. Instead, set h2UpgradePolicy to DO_NOT UPGRADE. Otherwise, the preceding 503 failure result is returned. This is why the access log contains "protocol":"HTTP/1.1".

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  labels:
    provider: asm
  name: websockets-server
spec:
  host: websockets-server
  trafficPolicy:
    connectionPool:
      http:
        h2UpgradePolicy: DO_NOT_UPGRADE

How to Support HTTP/2

Then, we run the preceding example under Istio 1.12 or later and set up the HTTP2 upgrade. It includes a two-part configuration:

1) Use the DestinationRule to set the HTTP2 upgrade for the target service websockets-server, which is to set h2UpgradePolicy: UPGRADE (as shown below):

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  labels:
    provider: asm
  name: websockets-server
spec:
  host: websockets-server
  trafficPolicy:
    connectionPool:
      http:
        h2UpgradePolicy: UPGRADE

2) HTTP/2 does not support WebSocket by default. However, Envoy allows you to tunnel transport WebSocket over HTTP/2 streams. This way, you can use a unified HTTP/2 network throughout the deployment process. Use EnvoyFilter to set allow_connect for the target workload, which is to set the protocol connection allowed for the upgrade of the allow_connect: true, as shown in the following figure.

The following EnvoyFitler resource object is generated. Please see the EnvoyFilter template page under the plug-in center for more information. Select Set allow_connect to true to allow upgraded protocol connections, specify Patch Context as SIDECAR_INBOUND, and bind the corresponding workload, which means the label is app: websockets-server.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: h2-upgrade-wss
  labels:
    asm-system: 'true'
    provider: asm
spec:
  workloadSelector:
    labels:
      app: websockets-server
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: SIDECAR_INBOUND
      proxy:
        proxyVersion: '^1\.*.*'
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          http2_protocol_options:
            allow_connect: true

After h2UpgradePolicy is enabled, the following log data is returned by the sidecar proxy on the client. As shown in the log, protocol means HTTP/1.1, indicating that the client initiates a request using the HTTP/1.1 protocol.

{"bytes_received":41,"requested_server_name":null,"x_forwarded_for":null,"downstream_local_address":"192.168.224.30:8080","duration":4151,"upstream_cluster":"outbound|8080||websockets-server.default.svc.cluster.local","path":"/","authority":"websockets-server.default.svc.cluster.local:8080","method":"GET","user_agent":"Python/3.9 websockets/10.2","route_name":"default","start_time":"2022-03-18T00:58:23.255Z","bytes_sent":25,"upstream_service_time":null,"protocol":"HTTP/1.1","istio_policy_status":null,"response_code":101,"trace_id":null,"downstream_remote_address":"10.208.0.117:60092","upstream_local_address":"10.208.0.117:60618","request_id":"8a6612d9-86d2-4b28-b6e2-0be0c98d9c1f","upstream_transport_failure_reason":null,"upstream_host":"10.208.0.116:80","response_flags":"UR"}{"method":"GET","upstream_local_address":"127.0.0.6:34477","protocol":"HTTP/2","upstream_host":"10.208.0.116:80","response_code":101,"authority":"websockets-server.default.svc.cluster.local:8080","istio_policy_status":null,"x_forwarded_for":null,"bytes_sent":25,"upstream_service_time":null,"upstream_transport_failure_reason":null,"request_id":"8a6612d9-86d2-4b28-b6e2-0be0c98d9c1f","start_time":"2022-03-18T00:58:23.258Z","downstream_remote_address":"10.208.0.117:60618","requested_server_name":"outbound_.8080_._.websockets-server.default.svc.cluster.local","trace_id":null,"downstream_local_address":"10.208.0.116:80","user_agent":"Python/3.9 websockets/10.2","route_name":"default","upstream_cluster":"inbound|80||","path":"/","bytes_received":41,"response_flags":"UC","duration":4147}

After you enable h2UpgradePolicy, the following log data is returned by the sidecar proxy on the server. As shown in the log, protocol means HTTP/2, which indicates the request protocol of the server has been upgraded to HTTP/2.

{"method":"GET","upstream_local_address":"127.0.0.6:34477","protocol":"HTTP/2","upstream_host":"10.208.0.116:80","response_code":101,"authority":"websockets-server.default.svc.cluster.local:8080","istio_policy_status":null,"x_forwarded_for":null,"bytes_sent":25,"upstream_service_time":null,"upstream_transport_failure_reason":null,"request_id":"8a6612d9-86d2-4b28-b6e2-0be0c98d9c1f","start_time":"2022-03-18T00:58:23.258Z","downstream_remote_address":"10.208.0.117:60618","requested_server_name":"outbound_.8080_._.websockets-server.default.svc.cluster.local","trace_id":null,"downstream_local_address":"10.208.0.116:80","user_agent":"Python/3.9 websockets/10.2","route_name":"default","upstream_cluster":"inbound|80||","path":"/","bytes_received":41,"response_flags":"UC","duration":4147}

Summary

In Istio 1.12 or later, through takingDestinationRule as the target service, set HTTP/2 upgrade and use EnvoyFilter to set the allow_connect enable switch for the target workload. Then, you can use the WebSocket over HTTP/2 protocol.

Community

Alibaba Cloud Service Mesh FAQ (1): How to Use the WebSocket over HTTP/2 Protocol

Background

Run under the HTTP/1.1 Protocol

HTTP2 Protocol Upgrade Issues

How to Support HTTP/2

Summary

Read previous post:

Read next post:

Xi Ning Wang(王夕宁)

You may also like

Comments

Xi Ning Wang(王夕宁)

Related Products

Cloud-Native Applications Management Solution

Function Compute

Container Service for Kubernetes

Alibaba Cloud Service Mesh