Practices for Building GitOps Delivery Based on ACK One and ACR

By Liusheng

Background

You can use a third-party Continuous Integration (CI) system to complete the following CI pipeline:

After the CI system pushes a container image of an application to an ACR image repository, the Continuous Delivery (CD) pipeline is triggered, and the container image of the application is updated. This example mainly demonstrates how ACK One GitOps automatically listens for ACR image repository changes. If there is a container mirror tags update that meets the filtering conditions, the following process will be triggered:

Limits

Only applies to apps created through the ACK One GitOps system
Only applies to applications hosted in the Git system using Kustomize or Helm orchestration
Only takes effect on applications that use the auto-sync update policy
The private image pull credential must be deployed in the same cluster as the GitOps system. You cannot read private image pull credentials across clusters.

Prerequisites

An ACK One multi-cluster management instance is created. Please visit this link for more information.
The GitOps feature is enabled for ACK One multi-cluster management instances. Please visit this link for more information.

Getting Started

The GitHub address of the sample application in this article: https://github.com/AliyunContainerService/gitops-demo.git

In this example, the application changes are written back to the Git repository. Therefore, you need to fork the Git repository to your account first. For example, this demonstration will use the following repository address: https://github.com/haoshuwei/gitops-demo.git

The directory structure of the gitops-demo application is:

.
├── manifests
│   └── helm
│       ├── Chart.yaml
│       ├── templates
│       │   ├── deployment.yaml
│       │   ├── _helpers.tpl
│       │   ├── hpa.yaml
│       │   ├── ingress.yaml
│       │   ├── NOTES.txt
│       │   ├── serviceaccount.yaml
│       │   ├── service.yaml
│       │   └── tests
│       │       └── test-connection.yaml
│       ├── values-oversea.yaml
│       └── values.yaml
└── README.md

In the values.yaml file, the image tag used to dynamically render the application lock by image.tag:

image:
  repository: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "v1.0"

Step 1: Create an Application in the GitOps System

Please see this link for more information about how to connect to and access the GitOps system.

Add a Git source repository. Please see this link for more information.

Application creation reference: https://www.alibabacloud.com/help/en/container-service-for-kubernetes/latest/gitops-management

(1) Create CLI: create application gitops-demo. Please check below for more information about the parameters:

argocd app create gitops-demo --repo https://github.com/AliyunContainerService/gitops-demo.git --path  manifests/helm --dest-namespace gitops --dest-server https://xx.xx.XX.XX:6443 --sync-policy none

argocd app sync gitops-demo

(2) Create UI: create application gitops-demo.

Please see the following two figures for more information about the parameters:

View the running status of the application:

Step 2: Configure an Automatic Application Update

When you push a new version of an application image to the ACR image repository, you want the GitOps system to automatically detect the new version and update it to the actual environment. Before you configure automatic application updates, you need to know the following content:

In the GitOps system, the function of triggering automatic application updates based on changes in ACR image repositories is the responsibility of the argocd-image-updater component. This component is currently in the alpha phase. We only recommend using it in the development and test environment. If you want to use it in the production environment, you must carefully evaluate it at your own risk.
An application created by the GitOps system generates an Application CR in the argocd namespace. You can add annotations to the Application CR to configure the application automatic update.
In the GitOps system, you must manually configure the logon username and password of the ACR image repository to monitor changes in the image repository.

(1) View the Application CR of the gitops-demo:

$ kubectl -n argocd get application gitops-demo -oyaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: gitops-demo
  namespace: argocd
spec:
  destination:
    namespace: gitops
    server: https://x.xxx.xxx.x:6443
  project: default
  source:
    helm:
      valueFiles:
      - values.yaml
    path: manifests/helm
    repoURL: https://github.com/haoshuwei/gitops-demo.git
    targetRevision: HEAD
  syncPolicy:
    automated: {}
    syncOptions:
    - CreateNamespace=true

(2) Add annotations for the gitops-demo

Generate imageUpdate.patch:

$ cat <<EOF > imageUpdate.patch
metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: echoserver=registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server
    argocd-image-updater.argoproj.io/echoserver.helm.image-tag: image.tag
    argocd-image-updater.argoproj.io/echoserver.update-strategy: semver
    argocd-image-updater.argoproj.io/echoserver.allow-tags: regexp:^v[1-9].*
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/git-creds
EOF

Add annotations to the gitops-demo:

$ kubectl -n argocd patch Application gitops-demo --type=merge -p "$(cat imageUpdate.patch)"

(3) Configure credentials for accessing ACR and Git repositories

Configure credentials for accessing ACR:

$ kubectl -n argocd apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: acr
type: Opaque
stringData:
  acr: <your_username>:<your_password>  # Replace <your_username>:<your_password> with the credentials of your image repository. 
EOF

Configure credentials for accessing the Git repository:

$ kubectl -n argocd create secret generic git-creds \
--from-literal=username=<your_username> \
--from-literal=password=<your_password>

Step 3: Test the Automatic Update of the Application

(1) Push the new image tag to the ACR image repository:

$ docker pull registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server:v1.0
$ docker tag registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server:v1.0 registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server:v2.0
$ docker push registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server:v2.0

(2) Run the following command to view the logs of argocd-image-updater containers in the argocd namespace:

$ kubectl -n argocd logs -f argocd-server-xxxxxxxxxx-xxxxx -c argocd-image-updater

Expected output:

time="2022-12-29T09:58:13Z" level=info msg="Successfully updated the live application spec" application=gitops-demo
time="2022-12-29T09:58:13Z" level=info msg="Processing results: applications=1 images_considered=1 images_skipped=0 images_updated=1 errors=0"

Check whether the application image in the destination cluster is automatically updated.

(3) Check whether the manifests/helm/.argocd-source-gitops-demo.yaml file is automatically generated in the Git repository on GitHub (indicating that the Git write-back is successful).

Configure the Automatic Update of Application Images

1. Configure to Update the Specified Container Image

You can set the annotation to mark one or more container images for an application created in the GitOps system to be automatically updated. You must set annotations in the following format:

argocd-image-updater.argoproj.io/image-list: <image_spec_list>

It can be a single container image or a group of container image lists separated with commas (,). You must specify container images in the following format:

[<alias_name>=]<image_path>[:<version_constraint>]

It is the container image alias of the application, so it can be referenced when making other configurations. Aliases can only contain alphanumeric strings and can only be used in the annotations of image-list. In this example, the image to be automatically updated is specified as registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server, and the alias set for it is echoserver:

argocd-image-updater.argoproj.io/image-list: echoserver=registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server

2. Set Filter Conditions for Image Tags

Let’s take registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server images as an example. You can set filter conditions to automatically update the application only when the echo-server images in the ACR image repository have new tags that meet the filter rules.

The following filter condition allows only tags that match the regular expression to trigger image updates:

argocd-image-updater.argoproj.io/<image_name>.allow-tags: <match_func>

The format is regexp:, which is the standard regular expression.

The following example describes the standard regular expression:

argocd-image-updater.argoproj.io/echoserver.allow-tags: regexp:^v[1-9].*

3. Set the Container Image Update Policy

The following table describes the container image update policies that you can configure. The default container image update policy is semver.

Update Policy	Description
semver	Update to the latest image version in a list that is sorted based on semantic versions
latest	Sort by creation time and update to the latest image tag (Note: Not the time when the image was pushed to the repository)
name	Update to the latest image version in an alphabetically sorted list
digest	Update to the latest image version pushed with a mutable tag

The annotation used to specify a container image update policy must be in the following format:

argocd-image-updater.argoproj.io/<image_name>.update-strategy: <strategy>

The following example shows that the image update policy used is semver:

argocd-image-updater.argoproj.io/echoserver.update-strategy: semver

4. Set Application Parameters for Helm/Kustomize Orchestration

Set Application Parameters for Helm Orchestration

An application may contain descriptions of multiple container images. For example, the values.yaml file applied by gitops-demo contains image.repository image.tag configurations. The following format describes the configurations of Annotations:

annotations:
  argocd-image-updater.argoproj.io/image-list: echoserver=registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server
  argocd-image-updater.argoproj.io/echoserver.helm.image-name: image.repository
  argocd-image-updater.argoproj.io/echoserver.helm.image-tag: image.tag

Set Application Parameters for Kustomize Orchestration

Image update annotation for applications whose manifests are rendered and managed by using Kustomize: You must specify the aliases of the new images and the addresses of the original images. You can include image tags in image aliases but cannot include image tags in image addresses. The annotation must be in the following format:

annotations:
  argocd-image-updater.argoproj.io/image-list: <image_alias>=<image_name>:<image_tag>
  argocd-image-updater.argoproj.io/<image_alias>.kustomize.image-name: <original_image_name>

Configure Credentials for Accessing an ACR Image Repository

You must configure the following access credential to use GitOps to automatically monitor changes in the ACR image repository and update the application. Configurations related to the ACR image repository are stored in a ConfigMap named argocd-image-updater-config in the argocd namespace.

View the default configuration through the following format:

$ kubectl -n argocd get cm argocd-image-updater-config -oyaml
apiVersion: v1
data:
  registries.conf: |
    registries:
    - name: AlibabaCloud Container Registry
      api_url: https://registry-vpc.<RegionID>.aliyuncs.com
      prefix: registry-vpc.<RegionID>.aliyuncs.com
      insecure: no
      credentials: secret:argocd/acr#acr
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-image-updater-config
    app.kubernetes.io/part-of: argocd-image-updater
  name: argocd-image-updater-config
  namespace: argocd

Parameter	Description
name	The name of the image repository
api_url	The API address of the container image repository. is automatically generated based on dynamic rendering in the current region.
prefix	The query prefix of the container image repository. The prefix is automatically generated based on dynamic rendering in the current region.
credentials	The query prefix of the container image repository. When the system installs GitOps, the system automatically generates the prefix based on the region where GitOps is installed.

If you are using an ACREE image repository, replace the values of the api_url and prefix fields.

According to the credentials: secret:argocd/acr#acr configuration, we need to create a secret named acr in the argocd namespace to store the credentials for accessing the ACR image repository:

kubectl -n argocd apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: acr
type: Opaque
stringData:
  # Replace <your_username >:< your_password> with your container image repository access credential.
  acr: <your_username>:<your_password>  
EOF

Configure Credentials for Accessing the Git Repository

If you configure the username and password when you add a Git Repository to the GitOps system, the application that uses the Git Repository has permission to write back the change information of the application container image to the Git system by default. You can add the following annotations to configure the Git credential:

annotations:
  argocd-image-updater.argoproj.io/write-back-method: git

If you need to customize the Git credential, you can use the specified Git credential by adding the following annotations:

annotations:
  argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/git-creds

And create a secret named git-creds under the argocd namespace:

$ kubectl -n argocd create secret generic git-creds \
--from-literal=username=<your_username> \
--from-literal=password=<your_password>

Please visit this link for more image-updater usage.

Community

Practices for Building GitOps Delivery Based on ACK One and ACR

Background

Limits

Prerequisites

Getting Started

Step 1: Create an Application in the GitOps System

Step 2: Configure an Automatic Application Update

Step 3: Test the Automatic Update of the Application

Configure the Automatic Update of Application Images

1. Configure to Update the Specified Container Image

2. Set Filter Conditions for Image Tags

3. Set the Container Image Update Policy

4. Set Application Parameters for Helm/Kustomize Orchestration

Configure Credentials for Accessing an ACR Image Repository

Configure Credentials for Accessing the Git Repository

Read previous post:

Read next post:

Alibaba Container Service

You may also like

Comments

Alibaba Container Service

Related Products

Best Practices

ACK One

Container Service for Kubernetes

Cloud-Native Applications Management Solution