Kruise Rollout v0.2.0: Support Gateway API and StatefulSet Rollout in Batches

Zhao Mingshan (Liheng)

Preface

Kruise Rollout [1] is an open-source progressive delivery framework from the OpenKruise community. Kruise Rollout supports canary rollout, blue-green rollout, and A/B Testing rollout in conjunction with traffic and instance grayscale. In the rollout process, Kruise Rollout can automatically perform batch rollout and pause the rollout automatically based on the Prometheus Metrics. It also provides non-inductive docking of the by-pass and compatibility with multiple existing workloads (Deployment and CloneSet).

Gateway API

Ingress API is an abstraction for service gateways in Kubernetes. It is also the most widely used gateway resource in the Kubernetes community, with the most representative being the Nginx Ingress Controller. However, there are also some problems with Ingress resources, mainly because the definition of Ingress is relatively simple and cannot meet some complex network requirements. In many scenarios, Ingress controllers need to be extended by defining Annotations or CRD. For example, Istio extends Virtual Service and DestinationRule resources.

The SIG-NETWORK community proposed the Gateway API resource to solve the problems above and promote the community to use unified standards, which is a collection of API resources in Kubernetes, including GatewayClass, Gateway, HTTPRoute, TCPRoute, and Service. Together, these resources build models for various network use cases. Currently, this interface has been implemented in many community open-source projects (such as Istio, Nginx, and Kong). As a progressive delivery framework, Kruise Rollout should support the Gateway API. The following is an example of using the Gateway API for canary rollout:

apiVersion: gateway.networking.Kubernetes.io/v1alpha2 kind: HTTPRoute metadata:   name: echoserver spec:   hostnames:   - test.app.domain   rules:   - backendRefs:     - group: ""       name: echoserver       port: 80 --- apiVersion: rollouts.kruise.io/v1alpha1 kind: Rollout spec:   objectRef:     ...   strategy:     canary:       steps:       - weight: 20         pause: {}       trafficRoutings:       - service: echoserver         gateway:           httpRouteName: echoserver

StatefulSet & Advanced StatefulSet Rollout in Batches

Kruise Rollout supports the batch rollout of stateless applications (Deployments and CloneSets) in Version 0.1.0, while stateful applications have similar demands. The StatefulSet community supports retaining the number of Pods in the old version during the rollout process (old-version Pods with an Order smaller than Partition should be retained), so Kruise Rollout can easily integrate stateful workloads based on this feature (including Advanced StatefulSet of Kruise). The following is an example of a rollout in three batches:

apiVersion: apps/v1 kind: StatefulSet metadata:   name: echoserver spec:   replicas: 5   template:     spec:       containers:         - name: echoserver           image: cilium/echoserver:latest --- apiVersion: rollouts.kruise.io/v1alpha1 kind: Rollout metadata:   name: rollouts-demo spec:   objectRef:     workloadRef:       apiVersion: apps/v1       kind: StatefulSet       name: echoserver   strategy:     canary:       steps:       - replicas: 1         pause: {}       - replicas: 2         pause: {duration: 60}       - replicas: 2

Rollout Batch Tagging Capability

Developers considered many ease-of-use issues of Kruise Rollout at the beginning of its design. It can be quickly integrated with many excellent deployment solutions in the community. For example, users can use Helm to complete the Rollout delivery of applications. With the increase of Kruise Rollout users and scale, new requirements for ease of use have been put forward, such as:

During the canary rollout, if some exceptions are found in the business monitoring. Kruise Rollout is expected to quickly filter out the first batch of Pods to troubleshoot the problem.
The product planning on the Container Platform is expected to include a rollout details page to display the Pods of each batch and the progress of Rollout accurately.

Kruise Rollout has added the Batch Tagging on Pod capability to meet the requirements above. In the Rollout process, each batch of Pods can be tagged with the corresponding batch label[apps.kruise.io/rollout-batch-id]={Value as the corresponding batch, such as 1,2,3...}. The usage is listed below:

apiVersion: rollouts.kruise.io/v1alpha1 kind: Rollout metadata:   name: rollouts-demo spec:   ...   # required   rolloutID: v1

RolloutID is the ID for each rollout. This field is entered by the upper-layer PaaS platform or users. It can be an arbitrary string and needs to be different before and after the two rollouts. For example, webserver-20220728120533. Why is rolloutID necessary? CloneSet supports in-situ upgrades. In this scenario, the rollout batch label on Pods may be from the last rollout. Therefore, CloneSet can be used together with rolloutID to tag any batch of this rollout.

KubeVela Implements Canary Rollout Based on Kruise Rollout

KubeVela [2] is a cloud-native application management platform based on the OAM model. It provides comprehensive application delivery, application distribution, and multi-cluster management capabilities. Currently, Kruise Rollout has been integrated into KubeVela, and the Helm Charts canary rollout can be easily implemented using trait. For more information, please see the doument[3] at the end of this article:

apiVersion: core.oam.dev/v1beta1 kind: Application spec:   components:   - name: canary-demo     type: webservice     properties:       image: barnett/canarydemo:v1     traits:     - type: kruise-rollout       properties:         canary:           steps:           # The first batch of Canary releases 20% Pods, and 20% traffic imported to the new version, require manual confirmation before subsequent releases are completed           - weight: 20             trafficRoutings:             - type: nginx

Conclusion

As a by-pass progressive delivery framework, Kruise Rollout can be easily integrated with excellent application delivery platforms in the community. Users don't need to make additional changes; they only need a Kruise Rollout CRD definition.

You are welcome to try it! If you encounter any problem, please submit an issue.