All Products
Search
Document Center

:Use ALB Ingresses to perform canary releases in an ACK cluster

Last Updated:Aug 23, 2024

Before you update a Service, you can use canary releases to test the stability of the new Service version. Application Load Balancer (ALB) Ingresses allow you to use canary annotations to perform canary releases based on headers, cookies, and weights. Canary releases that use different rules take effect in the following order: header-based > cookie-based > weight-based. If you configure header-based, cookie-based, and weight-based rules at the same time, the rules with the highest priority shall prevail.

Prerequisites

Usage notes

  • Header-based and cookie-based rules cannot be configured with weight-based rules on the same Ingress. You must configure them on two separate Ingresses or use custom routing rules.

  • When you use canary annotations to perform canary releases, the order in which the ALB Ingress routing rules take effect depends on the lexicographical order of Ingress namespaces or names. To ensure that the canary release rules are applied in the correct order, you can use the alb.ingress.kubernetes.io/order annotation to define the order. The valid values of alb.ingress.kubernetes.io/order are 1 to 1000. The default value is 10. The smaller the value, the higher the priority. For example, if you want to increase the priority of an Ingress, you can reduce the value of its alb.ingress.kubernetes.io/order setting.

  • You can use custom routing rules to perform canary releases based on more complex routing conditions. For more information, see Customize the routing rules of an ALB Ingress.

Step 1: Create an application

  1. Obtain the kubeconfig file of the cluster and use kubectl to connect to the cluster

  2. Deploy a Service named tea.

    1. Create a file named tea-deploy.yaml and copy the following content to the file:

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: tea
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: tea
        template:
          metadata:
            labels:
              app: tea
          spec:
            containers:
            - name: tea
              image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx:latest
              ports:
              - containerPort: 80
    2. Run the following command to deploy the tea Service:

      kubectl apply -f tea-deploy.yaml
  3. Deploy a Service named tea-svc.

    1. Create a file named tea-svc.yaml and copy the following content to the file:

      apiVersion: v1
      kind: Service
      metadata:
        name: tea-svc
      spec:
        ports:
        - port: 80
          targetPort: 80
          protocol: TCP
        selector:
          app: tea
        type: NodePort
    2. Run the following command to deploy the tea-svc Service:

      kubectl apply -f tea-svc.yaml
  4. Deploy an Ingress named tea-ingress.

    1. Create a file named tea-ingress.yaml and copy the following content to the file:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: tea-ingress
      spec:
        ingressClassName: alb
        rules:
         - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
           http:
            paths:
            - path: /
              pathType: Prefix
              backend:
                service:
                  name: tea-svc
                  port:
                    number: 80
    2. Run the following command to deploy the Ingress:

      kubectl apply -f tea-ingress.yaml

Step 2: Perform a canary release for a new Service version

Deploy a new Service version and a new Ingress to route all requests that carry the location: hz header to the new Service version named canary and route 50% of the other requests to the new Service version. Other requests include requests that carry other headers and requests that do not the carry header.

  1. Deploy the new Service version named canary.

    1. Create a file named canary-deploy.yaml and copy the following content to the file:

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: canary
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: canary
        template:
          metadata:
            labels:
              app: canary
          spec:
            containers:
            - name: canary
              image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx:latest
              ports:
              - containerPort: 80
    2. Run the following command to deploy the canary Service:

      kubectl apply -f canary-deploy.yaml
  2. Deploy a Service named canary-svc.

    1. Create a file named canary-svc.yaml and copy the following content to the file:

      apiVersion: v1
      kind: Service
      metadata:
        name: canary-svc
      spec:
        ports:
        - port: 80
          targetPort: 80
          protocol: TCP
        selector:
          app: canary
        type: NodePort
    2. Run the following command to deploy the canary-svc Service:

      kubectl apply -f canary-svc.yaml
  3. Deploy an Ingress to route requests based on headers.

    1. Create a file named canary-header-ingress.yaml and copy the following content to the file:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          alb.ingress.kubernetes.io/canary: "true"
          alb.ingress.kubernetes.io/canary-by-header: "location"
          alb.ingress.kubernetes.io/canary-by-header-value: "hz"
        name: canary-header-ingress
        namespace: default
      spec:
        ingressClassName: alb
        rules:
          - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
            http:
              paths:
                - backend:
                    service:
                      name: canary-svc
                      port:
                        number: 80
                  path: /
                  pathType: Prefix
      • Set alb.ingress.kubernetes.io/canary to true to enable canary annotations.

      • Set alb.ingress.kubernetes.io/canary-by-header and alb.ingress.kubernetes.io/canary-by-header-value to the key and value of the header that you want to match. In this example, the KV pair of the header is set to location: hz. All requests that carry the location: hz header are routed to the new Service version. Requests that carry other headers are matched against other canary release rules based on the priorities of the rules and then routed to the Service versions associated with the matching rules.

    2. Run the following command to deploy an Ingress to route requests based on headers:

      kubectl apply -f canary-header-ingress.yaml
  4. Deploy an Ingress to route requests based on weights.

    1. Create a file named canary-weight-ingress.yaml and copy the following content to the file:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          alb.ingress.kubernetes.io/canary: "true"
          alb.ingress.kubernetes.io/canary-weight: "50"
        name: canary-weight-ingress
        namespace: default
      spec:
        ingressClassName: alb
        rules:
          - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
            http:
              paths:
                - backend:
                    service:
                      name: canary-svc
                      port:
                        number: 80
                  path: /
                  pathType: Prefix

      alb.ingress.kubernetes.io/canary-weight: specifies the percentage of traffic routed to the new Service version. In this example, the value is set to 50, which indicates that 50% of the traffic is routed to the new Service version.

    2. Run the following command to deploy an Ingress to route requests based on weights:

      kubectl apply -f canary-weight-ingress.yaml
  5. Check whether the canary release is successful.

    1. Run the following command to query the IP address of the ALB instance:

      kubectl get ing

      Expected output:

      NAME                    CLASS   HOSTS                     ADDRESS                                              PORTS   AGE
      canary-header-ingress   alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      8m23s
      canary-weight-ingress   alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      8m16s
      tea-ingress             alb     demo.domain.ingress.top   alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com   80      7m5s
    2. Run the following command multiple times to send requests that carry the location: hz header to the Service:

      curl -H Host:demo.domain.ingress.top -H "location:hz" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      Expected output:

      new

      new is returned for the requests that carry the location: hz header. The requests that carry the location: hz header are routed to the new Service version.

    3. Run the following command multiple times to send requests that do not carry headers to the Service:

      curl -H Host:demo.domain.ingress.top  http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      new is returned for 50% of the requests that do not carry headers and old is returned for the remaining 50% of the requests. 50% of the requests that do not carry headers are routed to the new Service version.

    4. Run the following command multiple times to send requests that carry the location: bj header to the Service:

      curl -H Host:demo.domain.ingress.top -H "location:bj" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      new is returned for 50% of the requests that carry the location: bj header and old is returned for the remaining 50% of the requests. 50% of the requests that carry the location: bj header are routed to the new Service version.

    All requests that carry the location: hz header are routed to the new Service version named canary. Only 50% of the requests that carry other headers and do not carry headers are routed to the new Service version. The canary release is successful.

Step 3: Deprecate the old Service version

After the new Service version runs as expected for a period of time, you need to deprecate the old Service version and keep only the new Service version. To do this, you need to change the Service in the Ingress of the old Service version to the new Service version so that the Ingress can route traffic to the new Service version. Then, delete the canary Ingresses.

  1. Run the following command to modify the tea-ingress.yaml file:

    vim tea-ingress.yaml

    Run the following command to change the Service in the tea-ingress.yaml file from tea-svc to canary-svc:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: tea-ingress
    spec:
      ingressClassName: alb
      rules:
       - host: demo.domain.ingress.top # Replace with your domain name and ensure that it can be resolved to the IP address of the load balancer where the Ingress controller resides. 
         http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: canary-svc # Change tea-svc to canary-svc. 
                port:
                  number: 80
  2. Run the following command for the modified Ingress to take effect:

    kubectl apply -f tea-ingress.yaml
  3. Check whether the old Service version is deprecated.

    1. Run the following command multiple times to send requests that carry the location: hz header to the Service:

      curl -H Host:demo.domain.ingress.top -H "location:hz" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      Expected output:

      new

      new is returned for the requests that carry the location: hz header. Requests that carry the location: hz header are routed to the new Service version.

    2. Run the following command multiple times to send requests that do not carry headers to the Service:

      curl -H Host:demo.domain.ingress.top http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      Expected output:

      new

      new is returned for the requests that do not carry headers. Requests that do not carry headers are routed to the new Service version.

    3. Run the following command multiple times to send requests that carry the location: bj header to the Service:

      curl -H Host:demo.domain.ingress.top -H "location:bj" http://alb-ny3ute4r8yevni****.cn-chengdu.alb.aliyuncs.com

      Expected output:

      new

      new is returned for the requests that carry the location: bj header. Requests that carry the location: bj header are routed to the new Service version.

    Requests that carry the location: hz header, requests that carry other headers, and requests that do not carry headers are all routed to the new Service version. The old Service version is deprecated.

  4. Run the following command to delete the canary Ingresses named canary-weight-ingress and canary-header-ingress:

    kubectl delete ing canary-weight-ingress canary-header-ingress