×
Community Blog KubeVela-Based GitOps Delivery

KubeVela-Based GitOps Delivery

This article describes the steps of using KubeVela to deliver in GitOps mode.

By Dong Tianxin (Wuwu)
Auditing & Proofreading by Xiyang, Haizhu

KubeVela is a simple, easy-to-use, highly scalable cloud-native application management and delivery platform. It enables developers to define and deliver modern microservice applications on Kubernetes without knowing the details related to the Kubernetes infrastructure.

The OAM model behind KubeVela solves management problems, such as the combination and orchestration of complex resources during application building. It also sets a model for later O&M strategies. This means KubeVela can manage complex large-scale applications in combination with GitOps and simplify system complexity caused by team and system scale growth.

What Is GitOps?

The core idea of GitOps is to store the basic architecture and declarative description of the application configuration required by the application system in the Git warehouse and cooperate with an automated process. This way, the automated process can update the environment to the latest configuration each time the warehouse is updated.

This method allows developers to automatically deploy applications by changing the code and configuration in the Git warehouse. GitOps can bring the following advantages to application development:

  • Improve Production Efficiency: Automatic continuous deployment can speed up the average deployment and increase development efficiency
  • Lower the Threshold for Developers to Perform Deployment: GitOps pushes code instead of container configurations to enable developers to deploy without knowing the internal implementation of Kubernetes.
  • Make Change Records Traceable: If you use Git to manage clusters, every change is traceable, and the audit trail capability is strengthened.
  • You can use the rollback/branch feature of Git to restore the cluster.

KubeVela and GitOps

KubeVela, as a declarative application delivery and control plane, natively supports GitOps. It enables users to experience end-to-end application delivery and management and enjoy the following benefits brought by GitOps:

  • Application delivery workflow (CD pipeline)

    • KubeVela supports describing the procedural application delivery process in GitOps mode rather than briefly declaring the final status.
  • Process various dependencies and topologies during the deployment
  • Provide a unified upper-layer abstraction based on the semantics of various existing GitOps tools to simplify application delivery and management
  • Perform unified declaration, deployment, and service binding of cloud services
  • Provide out-of-the-box delivery strategies (including canary release and blue-green release)
  • Provide out-of-the-box hybrid/multi-cloud deployment strategies (including placement rules and cluster filtering rules)
  • Provide Kustomize-style Patch in a multi-environment delivery to describe deployment differences. Users do not have to learn the details of Kustomize.

This article describes the steps of using KubeVela to deliver in GitOps mode.

GitOps Workflow

GitOps workflow consists of CI (Continuous Integration) and CD (Continuous Delivery):

  • CI: Continuous integration builds code for business applications, builds images, and pushes them to the image repository. Multiple mature CI tools exist, such as GitHub Action and Travis (commonly used in open-source projects) and Jenkins and Tekton (commonly used in enterprises). In this article, GitHub Action is used to complete the CI step. You can use other CI tools instead. KubeVela can integrate with CI processes of all tools based on GitOps.
  • CD: Continuous deployment automatically updates the configurations in the cluster, such as updating the latest image from the image repository to the cluster. CD can be completed in one of the following schemes:
  1. Push-Based: CD in Push mode is completed by configuring the CI workflow. The Access Key pair of the cluster needs to be shared with CI so the CI workflow can push changes to the cluster through commands. Please refer to the blog entitled Using Jenkins + KubeVela to Complete the Continuous Delivery of Applications via the relevant links at the end of this article.
  2. Pull-Based: CD in Pull mode listens to the changes in the code repository or configuration repository in the cluster and synchronizes the changes to the cluster. Compared with the Push mode, the cluster proactively pulls updates in the Pull mode, avoiding key exposure. This is the core content of this article.

Applications are delivered to the following two types of personnel:

  1. Infrastructure Delivery to Platform Administrators and O&M Personnel: Users can update the configuration files in the repository to update the infrastructure configurations in the cluster, such as the dependent software, security policies, storage, and network of the system.
  2. Delivery to End Developers: Once the code of the user is merged into the application code repository, the application update is automatically triggered in the cluster. This improves the application iteration efficiency. Combined with the canary release, traffic transfer, and multi-cluster deployment features, KubeVela can provide a powerful automated release capability.

Delivery to Platform Administrators/O&M Personnel

In the following figure, platform administrators/O&M personnel do not need to know the application code. They only need to prepare a Git configuration repository and deploy KubeVela configuration files. Subsequent configuration modifications of applications and infrastructure can be completed by updating the Git configuration repository. This makes each configuration modification traceable.

1

Prepare Configuration Repository

Please see Sample Repository 1 for specific configuration via the related links at the end of this article.

In this example, we deploy MySQL database software as the infrastructure of the project and a business application to use this database. The sample directory structure of the configuration repository contains the following elements:

  • clusters/ contains the KubeVela GitOps configuration in the cluster. You need to manually deploy the files in clusters/ to the cluster. This is a one-time control operation. After the operation is performed, KubeVela automatically listens to file modifications in the configuration repository and updates the configurations in the cluster. The clusters/apps.yaml file listens to modifications in all applications in the apps/ directory, while the clusters/infra.yaml file listens to modifications in all infrastructures in the infrastructure/ directory.
  • The apps/ directory contains all the configurations of the business application. In this example, it is a business application that uses the database.
  • The infrastructure/ directory contains infrastructure-related configurations and policies. In this example, the infrastructure is the MySQL database.
├── apps
│   └── my-app.yaml
├── clusters
│   ├── apps.yaml
│   └── infra.yaml
└── infrastructure
    └── mysql.yaml

KubeVela recommends using the preceding directory structure to manage your GitOps repository. You can use the clusters/ directory to store related KubeVela GitOps configurations and manually deploy the configurations to the cluster. Use the apps/ directory and the infrastructure/ directory to store your application and infrastructure configurations. You can manage your deployment environment properly and isolate the impact of application changes by separating the application configuration from the infrastructure configuration.

clusters/ directory

The clusters directory is also the configuration directory of the initialization operation for KubeVela to connect to GitOps.

Take clusters/infra.yaml as an example:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: infra
spec:
  components:
  - name: database-config
    type: kustomize
    properties:
      repoType: git
      # replace this url with the address of the git configuration repository that you want to listen to
      url: https://github.com/FogDong/KubeVela-GitOps-Infra-Demo
      # If it is a private repository that you want to listen to, you must associate the url with a git secret.
      # secretRef: git-secret
      # The time interval for automatically pulling configuration. Because the infrastructure is usually slightly modified, set this parameter to 10 minutes.
      pullInterval: 10m
      git:
        # listen to the modified branch
        branch: main
      # listen to the modified path, and the path points to the files in the infrastructure directory in the repository
      path: ./infrastructure

apps. Yaml and infra.yaml are almost the same, but they listen to different file directories. In apps.yaml, the value of properties.path is set to ./apps, indicating that file modifications in the apps/ directory are listened to. The GitOps control configuration file in the

cluster folder must be manually deployed to the cluster at one time during initialization. After that, KubeVela automatically listens to the configuration files in the apps/ and infrastructure/ directories and updates and synchronizes them regularly.

apps/ directory

The apps/ directory stores application configuration files, which is a simple application configured with database information and Ingress. The application connects to a MySQL database and starts the service. In the default service path, the current version number is displayed. In the /db path, the information in the current database is listed.

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: my-app
  namespace: default
spec:
  components:
    - name: my-server
      type: webservice
      properties:
        image: ghcr.io/fogdong/test-fog:master-cba5605f-1632714412
        port: 8088
        env:
          - name: DB_HOST
            value: mysql-cluster-mysql.default.svc.cluster.local:3306
          - name: DB_PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: ROOT_PASSWORD
      traits:
        - type: ingress
          properties:
            domain: testsvc.example.com
            http:
              /: 8088

This application uses the webservice of the built-in component type of KubeVela. It is bound with Ingress O&M features. By declaring O&M capabilities in the application, you can collect the underlying Deployment, Service, and Ingress by using only one file. This facilitates easier application management.

infrastructure/ directory

The infrastructure/ directory stores infrastructure configurations. Here, we use a MySQL controller to deploy a MySQL cluster. (Please see the related links at the end of this article.)

Make sure you have a secret in your cluster and declare the MySQL password through ROOT_PASSWORD.

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: mysql
  namespace: default
spec:
  components:
    - name: mysql-controller
      type: helm
      properties:
        repoType: helm
        url: https://presslabs.github.io/charts
        chart: mysql-operator
        version: "0.4.0"
    - name: mysql-cluster
      type: raw
      dependsOn:
        - mysql-controller
      properties:
        apiVersion: mysql.presslabs.org/v1alpha1
        kind: MysqlCluster
        metadata:
          name: mysql-cluster
        spec:
          replicas: 1
          # associate the name of the secret
          secretName: mysql-secret

In the preceding sample MySQL application, the capabilities of the KubeVela workflow are used. The workflow is divided into two steps. The first step deploys the MySQL controller. After the controller is deployed and running properly, the second step deploys the MySQL cluster.

Deploy files in the clusters/ directory

After the files above are configured and stored in the Git configuration repository, you need to manually deploy the KubeVela GitOps configuration file in the clusters/ directory in the cluster.

First, deploy clusters/infra.yaml in the cluster. It automatically pulls up the MySQL deployment files in the infrastructure/ directory in the cluster.

kubectl apply -f clusters/infra.yaml
$ vela ls

APP     COMPONENT           TYPE        TRAITS  PHASE   HEALTHY STATUS  CREATED-TIME
infra   database-config     kustomize           running healthy         2021-09-26 20:48:09 +0800 CST
mysql   mysql-controller    helm                running healthy         2021-09-26 20:48:11 +0800 CST
└─      mysql-cluster       raw                 running healthy         2021-09-26 20:48:11 +0800 CST

Then, deploy clusters/apps.yaml in the cluster. It automatically pulls up the application deployment files in the apps/ directory.

kubectl apply -f clusters/apps.yaml
APP     COMPONENT           TYPE        TRAITS  PHASE   HEALTHY STATUS  CREATED-TIME
apps    apps                kustomize           running healthy         2021-09-27 16:55:53 +0800 CST
infra   database-config     kustomize           running healthy         2021-09-26 20:48:09 +0800 CST
my-app  my-server           webservice  ingress running healthy         2021-09-27 16:55:55 +0800 CST
mysql   mysql-controller    helm                running healthy         2021-09-26 20:48:11 +0800 CST
└─      mysql-cluster       raw                 running healthy         2021-09-26 20:48:11 +0800 CST

Applications and databases are automatically pulled up in the cluster by the deployment of the KubeVela GitOps configuration file.

The Ingress of curl shows that the current version is 0.1.5, and the application is connected to the database.

$ kubectl get ingress
NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   testsvc.example.com   <ingress-ip>    80      162m

$ curl -H "Host:testsvc.example.com" http://<ingress-ip>
Version: 0.1.5

$ curl -H "Host:testsvc.example.com" http://<ingress-ip>/db
User: KubeVela
Description: It's a test user

Modify Configurations

After the first deployment, you can update the configurations of applications in the cluster by modifying the configurations in the configuration repository.

The following code modifies the Domain of the application Ingress:

...
      traits:
        - type: ingress
          properties:
            domain: kubevela.example.com
            http:
              /: 8089

The following code shows the information about Ingress in the cluster after a while:

NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   kubevela.example.com  <ingress-ip>    80      162m

The Host address of Ingress has been updated.

This way, you can automatically update the configurations in a cluster with ease by updating the files in the Git configuration repository.

Delivery to End Developers

End developers need an application code repository in addition to the KubeVela Git configuration repository. In the following figure, after the code in the application code repository is updated, a CI needs to be configured to automatically build and push images to the image repository. KubeVela listens to the latest images in the image repository, automatically updates the image configurations in the configuration repository, and updates the application configurations in the cluster. This enables the configurations in the cluster to be automatically updated after the code is updated.

2

Prepare a Code Repository

Prepare a code repository that contains source code and the corresponding Dockerfile

The code connects to a MySQL database and starts the service. The current version number is displayed in the default service path. In the /db path, the information in the current database is listed.

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        _, _ = fmt.Fprintf(w, "Version: %s\n", VERSION)
    })
    http.HandleFunc("/db", func(w http.ResponseWriter, r *http.Request) {
        rows, err := db.Query("select * from userinfo;")
        if err != nil {
            _, _ = fmt.Fprintf(w, "Error: %v\n", err)
        }
        for rows.Next() {
            var username string
            var desc string
            err = rows.Scan(&username, &desc)
            if err != nil {
                _, _ = fmt.Fprintf(w, "Scan Error: %v\n", err)
            }
            _, _ = fmt.Fprintf(w, "User: %s \nDescription: %s\n\n", username, desc)
        }
    })

    if err := http.ListenAndServe(":8088", nil); err != nil {
        panic(err.Error())
    }

We hope that after the user modifies the code and submits it, the latest image is automatically built and pushed to the image repository. You can use GitHub Actions, Jenkins, or other CI tools to realize continuous integration. In this example, we use GitHub Actions. Please see Sample Repository 2 for more information about code files and configurations via the related links at the end of this article.

Key Information Configuration

After a new image is pushed to the image repository, KubeVela identifies the new image and updates the application configuration files in the repository and the cluster. Therefore, we need a Secret that contains Git information to enable KubeVela to submit to the Git repository. Deploy the following file and replace the username and password with your Git username and password (or Token):

apiVersion: v1
kind: Secret
metadata:
  name: git-secret
type: kubernetes.io/basic-auth
stringData:
  username: <your username>
  password: <your password>

Prepare Configuration Repository

The configuration repository is similar to the repository for O&M personnel. You only need to add the configurations related to the image repository. Please see Sample Repository 1 for specific configurations via the related links at the end of this article.

Modify apps.yaml in the clusters/ directory. This GitOps configuration listens to application file modifications in the apps/ directory in the repository and image updates in the image repository:

...
  imageRepository:
    # The URL of the image
    image: <your image>
    # If the image repository is a private one, you can create the corresponding image key and associate the key to the repository by using kubectl create secret docker-registry.
    # secretRef: imagesecret
    filterTags:
      # Support filtering by image tags.
      pattern: '^master-[a-f0-9]+-(?P<ts>[0-9]+)'
      extract: '$ts'
    # filter the latest image tags by using a policy and update the image
    policy:
      numerical:
        order: asc
    # append submission information
    commitMessage: "Image: {{range .Updated.Images}}{{println .}}{{end}}"

Modify the image field in apps/my-app.yaml and add the comment # {"$imagepolicy": "default:apps"}. KubeVela uses this comment to update the corresponding image fields. default:apps is the namespace and name corresponding to the preceding GitOps configuration.

spec:
  components:
    - name: my-server
      type: webservice
      properties:
        image: ghcr.io/fogdong/test-fog:master-cba5605f-1632714412 # {"$imagepolicy": "default:apps"}

After you update the files that contain the image repository configurations in the clusters/ directory to the cluster, you can update the application by modifying the code.

Modify Code

Change the version in the code file to 0.1.6 and modify data in the database:

const VERSION = "0.1.6"

...

func InsertInitData(db *sql.DB) {
    stmt, err := db.Prepare(insertInitData)
    if err != nil {
        panic(err)
    }
    defer stmt.Close()

    _, err = stmt.Exec("KubeVela2", "It's another test user")
    if err != nil {
        panic(err)
    }
}

Submit the modification to the code repository. The CI pipeline you configure starts to build images and pushes them to the image repository.

KubeVela listens to the image repository and updates the application my-app in the apps/ directory in the configuration repository based on the latest image tag.

In the process, kubevelabot has a commit in the configuration repository, and the commit information is prefixed with Update image automatically. You can also use {{range .Updated.Images}}{{println .}}{{end}} to append the information that you need to the commitMessage field.

3

Note: If you want to put the code and the configuration in the same repository, you need to filter out commits from kubevelabot to prevent the repeated building of the pipeline. You can achieve that by using the following configuration in CI:

jobs:
publish:
  if: "!contains(github.event.head_commit.message, 'Update image automatically')"

If you check the application in the cluster again, the image of the application my-app has been updated.

KubeVela obtains the latest information from the configuration repository and the image repository at regular intervals based on the specified interval.

  • If the configuration file in the Git repository is updated, KubeVela updates the applications in the cluster based on the latest configuration.
  • If new tags are added to the image repository, KubeVela filters out the latest image tags based on the policy rules you configure and updates the images to the Git repository. If the files in the code repository are updated, KubeVela repeats the first step to update the files in the cluster, thus realizing automatic deployment.

Use the Ingress corresponding to curl to view the current version and database information:

$ kubectl get ingress
NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   kubevela.example.com  <ingress-ip>    80      162m

$ curl -H "Host:kubevela.example.com" http://<ingress-ip>
Version: 0.1.6

$ curl -H "Host:kubevela.example.com" http://<ingress-ip>/db
User: KubeVela
Description: It's a test user

User: KubeVela2
Description: It's another test user

The version has been updated successfully. We have performed all the operations required, from changing the code to automatic deployment to the cluster.

Summary

On the O&M side, if you need to update the configurations of an infrastructure (such as the database) or the configuration items of an application, you only need to modify the files in the configuration repository. KubeVela automatically synchronizes the configurations to the cluster. The deployment process is simplified.

On the R&D side, after you modify the code in the code repository, KubeVela automatically updates the image in the configuration repository. Then, the version updates of an application are facilitated.

KubeVela accelerates the entire process from development to deployment of applications through a combination with GitOps.

References

  1. Using Jenkins + KubeVela to Complete Continuous Delivery of the Application: https://kubevela.io/blog/2021/09/02/kubevela-jenkins-cicd
  2. Sample Repository 1: https://github.com/oam-dev/samples/tree/master/09.GitOps_Demo/for-SREs
  3. MySQL Controller: https://github.com/bitpoke/mysql-operator
  4. Sample Repository 2: https://github.com/oam-dev/samples/tree/master/09.GitOps_Demo/for-developers
0 0 0
Share on

You may also like

Comments