Use ALB Ingresses to perform canary releases - Container Service for Kubernetes

Before you update a Service, you can use canary releases to test the stability of the new Service version. Application Load Balancer (ALB) Ingresses allow you to use canary annotations to perform canary releases based on headers, cookies, and weights. Canary releases that use different rules take effect in the following order: header-based > cookie-based > weight-based. If you configure header-based, cookie-based, and weight-based rules at the same time, the rules with the highest priority shall prevail.

Usage notes

Header-based and cookie-based rules cannot be configured with weight-based rules on the same Ingress. You must configure them on two separate Ingresses or use custom routing rules.
When you use canary annotations to perform canary releases, the order in which the ALB Ingress routing rules take effect depends on the lexicographical order of Ingress namespaces or names. To ensure that the canary release rules are applied in the correct order, you can use the alb.ingress.kubernetes.io/order annotation to define the order. The valid values of alb.ingress.kubernetes.io/order are 1 to 1000. The default value is 10. The smaller the value, the higher the priority. For example, if you want to increase the priority of an Ingress, you can reduce the value of its alb.ingress.kubernetes.io/order setting.
You can use custom routing rules to perform canary releases based on more complex routing conditions. For more information, see Customize the routing rules of an ALB Ingress.

Step 1: Create an application

Obtain the kubeconfig file of the cluster and use kubectl to connect to the cluster.

Deploy a Service named tea.

Create a file named tea-deploy.yaml and copy the following content to the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tea
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tea
  template:
    metadata:
      labels:
        app: tea
    spec:
      containers:
      - name: tea
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx:latest
        ports:
        - containerPort: 80

Run the following command to deploy the tea Service:
```
kubectl apply -f tea-deploy.yaml
```

Deploy a Service named tea-svc.

Create a file named tea-svc.yaml and copy the following content to the file:

apiVersion: v1
kind: Service
metadata:
  name: tea-svc
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: tea
  type: NodePort

Run the following command to deploy the tea-svc Service:
```
kubectl apply -f tea-svc.yaml
```

Deploy an Ingress named tea-ingress.

Create a file named tea-ingress.yaml and copy the following content to the file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tea-ingress
spec:
  ingressClassName: alb
  rules:
   - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
     http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tea-svc
            port:
              number: 80

Run the following command to deploy the Ingress:
```
kubectl apply -f tea-ingress.yaml
```

Step 2: Perform a canary release for a new Service version

Deploy a new Service version and a new Ingress to route all requests that carry the location: hz header to the new Service version named canary and route 50% of the other requests to the new Service version. Other requests include requests that carry other headers and requests that do not the carry header.

Deploy the new Service version named canary.

Create a file named canary-deploy.yaml and copy the following content to the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: canary
  template:
    metadata:
      labels:
        app: canary
    spec:
      containers:
      - name: canary
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx:latest
        ports:
        - containerPort: 80

Run the following command to deploy the canary Service:
```
kubectl apply -f canary-deploy.yaml
```

Deploy a Service named canary-svc.

Create a file named canary-svc.yaml and copy the following content to the file:

apiVersion: v1
kind: Service
metadata:
  name: canary-svc
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: canary
  type: NodePort

Run the following command to deploy the canary-svc Service:
```
kubectl apply -f canary-svc.yaml
```

Deploy an Ingress to route requests based on headers.
1. Create a file named canary-header-ingress.yaml and copy the following content to the file:
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/canary: "true"
    alb.ingress.kubernetes.io/canary-by-header: "location"
    alb.ingress.kubernetes.io/canary-by-header-value: "hz"
  name: canary-header-ingress
  namespace: default
spec:
  ingressClassName: alb
  rules:
    - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
      http:
        paths:
          - backend:
              service:
                name: canary-svc
                port:
                  number: 80
            path: /
            pathType: Prefix
```
  - Set alb.ingress.kubernetes.io/canary to true to enable canary annotations.
  - Set alb.ingress.kubernetes.io/canary-by-header and alb.ingress.kubernetes.io/canary-by-header-value to the key and value of the header that you want to match. In this example, the KV pair of the header is set to location: hz. All requests that carry the location: hz header are routed to the new Service version. Requests that carry other headers are matched against other canary release rules based on the priorities of the rules and then routed to the Service versions associated with the matching rules.
2. Run the following command to deploy an Ingress to route requests based on headers:
```
kubectl apply -f canary-header-ingress.yaml
```

Deploy an Ingress to route requests based on weights.

Create a file named canary-weight-ingress.yaml and copy the following content to the file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/canary: "true"
    alb.ingress.kubernetes.io/canary-weight: "50"
  name: canary-weight-ingress
  namespace: default
spec:
  ingressClassName: alb
  rules:
    - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
      http:
        paths:
          - backend:
              service:
                name: canary-svc
                port:
                  number: 80
            path: /
            pathType: Prefix

alb.ingress.kubernetes.io/canary-weight: specifies the percentage of traffic routed to the new Service version. In this example, the value is set to 50, which indicates that 50% of the traffic is routed to the new Service version.

Run the following command to deploy an Ingress to route requests based on weights:
```
kubectl apply -f canary-weight-ingress.yaml
```

Check whether the canary release is successful.
1. Run the following command to query the IP address of the ALB instance:
```
kubectl get ing
```
  Expected output:
```
NAME                    CLASS   HOSTS                     ADDRESS                                              PORTS   AGE
canary-header-ingress   alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      8m23s
canary-weight-ingress   alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      8m16s
tea-ingress             alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      7m5s
```
2. Run the following command multiple times to send requests that carry the location: hz header to the Service:
```
curl -H Host:demo.domain.ingress.top -H "location:hz" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  Expected output:
```
new
```
  new is returned for the requests that carry the location: hz header. The requests that carry the location: hz header are routed to the new Service version.
3. Run the following command multiple times to send requests that do not carry headers to the Service:
```
curl -H Host:demo.domain.ingress.top  http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  new is returned for 50% of the requests that do not carry headers and old is returned for the remaining 50% of the requests. 50% of the requests that do not carry headers are routed to the new Service version.
4. Run the following command multiple times to send requests that carry the location: bj header to the Service:
```
curl -H Host:demo.domain.ingress.top -H "location:bj" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  new is returned for 50% of the requests that carry the location: bj header and old is returned for the remaining 50% of the requests. 50% of the requests that carry the location: bj header are routed to the new Service version.
All requests that carry the location: hz header are routed to the new Service version named canary. Only 50% of the requests that carry other headers and do not carry headers are routed to the new Service version. The canary release is successful.

Step 3: Deprecate the old Service version

After the new Service version runs as expected for a period of time, you need to deprecate the old Service version and keep only the new Service version. To do this, you need to change the Service in the Ingress of the old Service version to the new Service version so that the Ingress can route traffic to the new Service version. Then, delete the canary Ingresses.

Run the following command to modify the tea-ingress.yaml file:

vim tea-ingress.yaml

Run the following command to change the Service in the tea-ingress.yaml file from tea-svc to canary-svc:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tea-ingress
spec:
  ingressClassName: alb
  rules:
   - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
     http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary-svc # Change tea-svc to canary-svc. 
            port:
              number: 80

Run the following command for the modified Ingress to take effect:
```
kubectl apply -f tea-ingress.yaml
```
Check whether the old Service version is deprecated.
1. Run the following command multiple times to send requests that carry the location: hz header to the Service:
```
curl -H Host:demo.domain.ingress.top -H "location:hz" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  Expected output:
```
new
```
  new is returned for the requests that carry the location: hz header. Requests that carry the location: hz header are routed to the new Service version.
2. Run the following command multiple times to send requests that do not carry headers to the Service:
```
curl -H Host:demo.domain.ingress.top http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  Expected output:
```
new
```
  new is returned for the requests that do not carry headers. Requests that do not carry headers are routed to the new Service version.
3. Run the following command multiple times to send requests that carry the location: bj header to the Service:
```
curl -H Host:demo.domain.ingress.top -H "location:bj" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com
```
  Expected output:
```
new
```
  new is returned for the requests that carry the location: bj header. Requests that carry the location: bj header are routed to the new Service version.
Requests that carry the location: hz header, requests that carry other headers, and requests that do not carry headers are all routed to the new Service version. The old Service version is deprecated.
Run the following command to delete the canary Ingresses named canary-weight-ingress and canary-header-ingress:
```
kubectl delete ing canary-weight-ingress canary-header-ingress
```