Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Phased Release of Application Services through Kubernetes Ingress Controller

Phased Release of Application Services through Kubernetes Ingress Controller

In this article, we discuss the phased release of application services through a Kubernetes ingress controller of Container Service.

Oftentimes, we need to upgrade services in our daily wok. Release methods that we usually use include rolling upgrade, batched suspension and release, blue-green release, and phased release. Today, we are going to show you how to implement the phased release and A/B testing of application services through an Ingress Controller in a Kubernetes cluster of Alibaba Cloud Container Service.

Release Scenarios

Scenario 1

Assume that in the current online environment, we have deployed Service A to provide layer-7 external services. Then, we have developed some new features and need to release a new-version Service A' in phased release mode. However, we do not want to simply and directly replace Service A. We want to forward client requests that contain foo=bar in the request header or cookie to Service A' at first. After Service A' is running stably for a period of time and all traffic is switched to Service A', we can smoothly discontinue Service A.

1

Scenario 2

Assume that in the current online environment, we have deployed Service B to provide layer-7 external services. Then, we have fixed some problems and need to release a new-version Service B' in phased release mode. However, we do not want to simply and directly switch the traffic of all clients to Service B'. We want to switch only 20% of traffic to Service B' at first. After Service B' is running stably for a period of time and all traffic is switched to Service B', we can smoothly discontinue Service B.

2

To meet different application release requirements, the Kubernetes Ingress Controller of Container Service supports multiple traffic splitting methods:

  1. Traffic splitting based on the request header, which is applicable to phased release and A/B testing scenarios.
  2. Traffic splitting based on the cookie, which is applicable to phased release and A/B testing scenarios.
  3. Traffic splitting based on the query parameter, which is applicable to phased release and A/B testing scenarios.
  4. Traffic splitting based on the service weight, which is applicable to blue-green release scenarios.

Annotation Description

The Kubernetes Ingress Controller of Container Service supports the phased release mechanism for application services through the following annotations:

1. nginx.ingress.kubernetes.io/service-match

This annotation is used to configure the routing matching rule of a new-version service. The format is shown in the following figure.

nginx.ingress.kubernetes.io/service-match: | 
    <service-name>: <match-rule>
# Parameter description:
# service-name: indicates the service name. Requests that conform to the rule specified by match-rule are routed to this service.
# match-rule: indicates the routing matching rule.
#
# Routing matching rule:
# 1. Supported matching types:
# - header: indicates matching based on the request header, which can be regular expression matching or exact expression matching.
# - cookie: indicates matching based on the cookie, which can be regular expression matching or exact expression matching.
# - query: indicates matching based on the query parameter, which can be regular expression matching or exact expression matching.
#
# 2. Matching modes:
# - Regular expression matching format: /{regular expression}/. // indicates matching based on a regular expression.
# - Exact expression matching format: "{exact expression}". "" indicates matching based on an exact expression.

The following figure shows some configuration examples of routing matching rules.

# Requests whose foo in the request header contains ^bar$ (regular expression matching) are forwarded to the new-nginx service.
new-nginx: header("foo", /^bar$/)

# Requests whose foo in the request header equals bar (exact expression matching) are forwarded to the new-nginx service.
new-nginx: header("foo", "bar")

# Requests whose foo in the cookie contains ^sticky-.+$ (regular expression matching) are forwarded to the new-nginx service.
new-nginx: cookie("foo", /^sticky-.+$/)

# Requests whose foo in the query parameter equals bar (exact expression matching) are forwarded to the new-nginx service.
new-nginx: query("foo", "bar")

2. nginx.ingress.kubernetes.io/service-weight

This annotation is used to configure a traffic weight for both old-version and new-version services. The format is shown in the following figure.

nginx.ingress.kubernetes.io/service-weight: | 
    <new-svc-name>:<new-svc-weight>, <old-svc-name>:<old-svc-weight>
Parameter description:
new-svc-name: indicates the name of the new-version service.
new-svc-weight: indicates the weight of the new-version service.
old-svc-name: indicates the name of the old-version service.
old-svc-weight: indicates the weight of the old-version service.

The following figure shows a service weight configuration example.

nginx.ingress.kubernetes.io/service-weight: | 
    new-nginx: 20, old-nginx: 60

Notes:

  1. A service weight is calculated based on the relative value. In the preceding example, the weight percentage of the new-nginx service is 25%, whereas that of the old-nginx service is 75%.
  2. In a service group (in which services have the same host and path in the Ingress YAML file), the default weight value of services without a specified weight is 100.

Version Updates

Currently, the Kubernetes Ingress Controller of Container Service must be upgraded to version 0.12.0-5 or later to support the traffic splitting feature. You can run the following commands to check the current version of the Kubernetes Ingress Controller.

  1. Deployed using Deployment
     kubectl -n kube-system get deploy nginx-ingress-controller -o yaml | grep -v 'apiVersion' | grep 'aliyun-ingress-controller'

  2. Deployed using DaemonSet
     kubectl -n kube-system get ds nginx-ingress-controller -o yaml | grep -v 'apiVersion' | grep 'aliyun-ingress-controller'


If the current version of the Kubernetes Ingress Controller in your cluster is earlier than 0.12.0-5, you can run the following commands to upgrade it to the new version.

Deploying using Deployment

 kubectl -n kube-system set image deploy/nginx-ingress-controller nginx-ingress-controller=registry.cn-hangzhou.aliyuncs.com/acs/aliyun-ingress-controller:0.12.0-5

Deploying using DaemonSet

 kubectl -n kube-system set image ds/nginx-ingress-controller nginx-ingress-controller=registry.cn-hangzhou.aliyuncs.com/acs/aliyun-ingress-controller:0.12.0-5

Then, the Kubernetes Ingress Controller in your cluster can support phased release.

Service Deployment

Before the phased release of applications, we must ensure that a set of applications has been deployed to provide external services. To this end, we can deploy an old-version service (old-nginx) and enable it to provide layer-7 domain name access through a Kubernetes Ingress Controller.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: old-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-nginx
  template:
    metadata:
      labels:
        run: old-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/xianlu/old-nginx
        imagePullPolicy: Always
        name: old-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: old-nginx
  sessionAffinity: None
  type: NodePort

Deploy Ingress resources.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # Old-version service
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80

Deploy the old-nginx service and test service access.

 kubectl get ing
NAME           HOSTS             ADDRESS        PORTS     AGE
gray-release   www.example.com   47.107.20.35   80        1m
 curl -H "Host: www.example.com"  http://47.107.20.35
old
 curl -H "Host: www.example.com"  http://47.107.20.35
old
 curl -H "Host: www.example.com"  http://47.107.20.35
old

Phased Release of the new-version Service

We need to release a new-version service (new-nginx) and configure it to allow only a portion of clients to access this service.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: new-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-nginx
  template:
    metadata:
      labels:
        run: new-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/xianlu/new-nginx
        imagePullPolicy: Always
        name: new-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: new-nginx
  sessionAffinity: None
  type: NodePort

Scenario 1: Set a specific rule to allow clients that conform to this rule to access the new-nginx service.

Assume that we want to route client requests that contain foo=bar in the request header to the new-nginx service. We can modify Ingress rules, as shown in the following figure.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    # Requests whose foo in the request header contains bar (regular expression matching) are routed to the new-nginx service.
    nginx.ingress.kubernetes.io/service-match: | 
      new-nginx: header("foo", /^bar$/)
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # Old-version service
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      # New-version service
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Test client access.

 curl -H "Host: www.example.com"  http://47.107.20.35
old
 curl -H "Host: www.example.com"  http://47.107.20.35
old
 curl -H "Host: www.example.com"  http://47.107.20.35
old
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
new
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
new
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
new

Scenario 2: Set a specific rule to route a specified proportion of requests that conform to this rule to the new-nginx service.

Assume that we want to route only 50% of the client requests that contain foo=bar in the request header to the new-nginx service. We can modify Ingress rules, as shown in the following figure.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    # Requests whose foo in the request header contains bar (regular expression matching) are routed to the new-nginx service.
    nginx.ingress.kubernetes.io/service-match: |
        new-nginx: header("foo", /^bar$/)
    # Only 50% of the requests that conform to the preceding matching rule are routed to the new-nginx service.
    nginx.ingress.kubernetes.io/service-weight: |
        new-nginx: 50, old-nginx: 50
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # Old-version service
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      # New-version service
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Test client access.

 curl -H "Host: www.example.com" http://47.107.20.35
old
 curl -H "Host: www.example.com" http://47.107.20.35
old
 curl -H "Host: www.example.com" http://47.107.20.35
old
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
new
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
old
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
old
 curl -H "Host: www.example.com" -H "foo: bar" http://47.107.20.35
new

Scenario 3: Route a specified proportion of requests to the new-nginx service.

Assume that we want to route only 50% of traffic to the new-nginx service. We can modify Ingress rules, as shown in the following figure.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
      # Only 50% of traffic is routed to the new-nginx service.
      nginx.ingress.kubernetes.io/service-weight: |
          new-nginx: 50, old-nginx: 50
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # Old-version service
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      # New-version service
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Test client access.

 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
old
 curl -H "Host: www.example.com" http://47.107.20.35
old
 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
old

Special note: If we do not configure the service-match or service-weight annotation, the Kubernetes Ingress Controller forwards requests evenly and randomly to the old-version and new-version services by default.

Completion of the Phased Release

After the new-version service is running stably for a period of time and meets our expectations, we need to discontinue the old-version service and keep only the new-version service online. At this point, we need to modify Ingress rules and delete relevant annotations and the old-version service. The ultimate rules are shown in the following figure.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # New-version service
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Test client access.

 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
new
 curl -H "Host: www.example.com" http://47.107.20.35
new

We can see that all requests are routed to the new-version service. Then, a phased release has been completed. At last, you can delete the old-version service and deployment.

Container Blockchain Computing Kubernetes GPU Metrics Auto Scaling
0 0 0
Share on

Read previous post:

Deploying Multiple Ingress Controllers in a Kubernetes Cluster

Read next post:

Analysis on Health Check Logic of a Kubernetes Ingress Controller

Alibaba Container Service

175 posts | 31 followers

You may also like

Comments

Alibaba Container Service

175 posts | 31 followers

Related Products

More Posts by Alibaba Container Service

See All