在CI Pipeline中Clone私有Git仓库 - 容器服务 Kubernetes 版 ACK

ACK One工作流集群构建CI Pipeline，其使用BuildKit Cache和NAS存储Go mode cache，可大幅加速CI Pipeline的流程。通过工作流集群构建基于Golang项目CI Pipeline时，若您使用的Git仓库为私有仓库，您需要在CI流程中先成功Clone该私有仓库，再进行CI Pipeline的构建操作。本文为您介绍如何在CI Pipeline中Clone私有Git仓库。

背景信息

使用公共Git仓库构建CI Pipeline的最佳实践，请参见基于工作流集群构建Golang项目的CI Pipeline。

若您使用私有Git仓库，则需要在上述最佳实践操作前，先Clone私有Git仓库。

本文为您提供以下三种方法Clone私有Git仓库：

方法一：基于Argo Workflows Git Artifact与用户名密码
方法二：基于Argo Workflows Git Artifact与SSH Private Key
方法三：基于Git Clone命令与用户名密码

在工作流集群中保存私有仓库凭据

Clone私有仓库前，您需要先在工作流集群中执行如下命令保存私有仓库所需的用户名、密码和ssh private key。

username、password和ssh-private-key需要替换为您实际使用的参数值。

kubectl create secret generic git-creds --from-literal="username=${username}" --from-literal="password=${password or token}" --from-file=ssh-private-key=${ssh private key path}

# example
# kubectl create secret generic git-creds --from-literal="username=demo" --from-literal="password=ghp_GePB****************d407" --from-file=ssh-private-key=$HOME/.ssh/id_rsa

方法一：基于Argo Workflows Git Artifact与用户名密码

该方法主要是在执行构建CI Pipeline的操作前，先执行Git Clone私有仓库操作，再进行Git Checkout操作。

以下YAML为了方便展示，和上文预置工作流模板内容相比，仅保留了上述CI Pipeline中的git-checkout-pr任务（其他方法均相同），基于此增加git-clone任务，并设置git-checkout-pr依赖git-clone。

git-checkout-pr的command中，shell script无需修改。
git-clone的artifacts中引用保存的私有仓库凭据的git-creds secret中的用户名、密码。

示例模板

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: ci-git-artifact
spec:
  entrypoint: main
  volumes:
  - name: run-test
    emptyDir: {}
  - name: workdir
    persistentVolumeClaim:
      claimName: pvc-nas
  - name: docker-config
    secret:
      secretName: docker-config
  arguments:
    parameters:
    - name: repo_url
      value: ""
    - name: repo_name
      value: ""
    - name: target_branch
      value: "main"
  templates:
    - name: main
      dag:
        tasks:
          - name: git-clone
            arguments:
              artifacts:
              - name: git-repo
                path: /workdir
                git:
                  repo: "{{arguments.parameters.repo_url}}"
                  revision: main
                  usernameSecret:
                    name: git-creds
                    key: username
                  passwordSecret:
                    name: git-creds
                    key: password
                  sshPrivateKeySecret:
                    name: git-creds
                    key: ssh-private-key
            inline:
              container:
                image: golang:1.10
                command: 
                - sh
                - -c
                - |
                  cd {{workflow.parameters.repo_name}}
                  git status && ls
                workingDir: /workdir
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
          - name: git-checkout-pr
            inline:
              container:
                image: alpine:latest
                command:
                  - sh
                  - -c
                  - |
                    set -eu
                    
                    apk --update add git
          
                    cd /workdir
                    echo "Start to Clone "{{workflow.parameters.repo_url}}
                    git -C "{{workflow.parameters.repo_name}}" pull || git clone {{workflow.parameters.repo_url}} 
                    cd {{workflow.parameters.repo_name}}
          
                    echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
                    git checkout {{workflow.parameters.target_branch}}
                    
                    echo "Get commit id" 
                    git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
                    commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
                    echo "Commit id is got: "$commitId
                                        
                    echo "Git Clone and Checkout Complete."
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
                resources:
                  requests:
                    memory: 1Gi
                    cpu: 1
                activeDeadlineSeconds: 1200
            depends: git-clone

提交Workflow参数说明

涉及参数和CI Pipeline保持一致，如下图所示：

方法二：基于Argo Workflows Git Artifact与SSH Private Key

和方法一基本相同，主要差异如下：

git-clone的artifacts中引用保存的私有仓库凭据的git-creds secret中的ssh private key。

在提交Workflow时，repo_url需要为ssh格式，例如：git@github.com:ivan-cai/gitops-demo-private.git。

示例模板

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: ci-git-artifact-sshkey
spec:
  entrypoint: main
  volumes:
  - name: run-test
    emptyDir: {}
  - name: workdir
    persistentVolumeClaim:
      claimName: pvc-nas
  - name: docker-config
    secret:
      secretName: docker-config
  arguments:
    parameters:
    - name: repo_url
      value: ""
    - name: repo_name
      value: ""
    - name: target_branch
      value: "main"
  templates:
    - name: main
      dag:
        tasks:
          - name: git-clone
            arguments:
              artifacts:
              - name: git-repo
                path: /workdir
                git:
                  repo: "{{arguments.parameters.repo_url}}"
                  revision: main
                  sshPrivateKeySecret:
                    name: git-creds
                    key: ssh-private-key
            inline:
              container:
                image: golang:1.10
                command: 
                - sh
                - -c
                - |
                  cd {{workflow.parameters.repo_name}}
                  git status && ls
                workingDir: /workdir
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
          - name: git-checkout-pr
            inline:
              container:
                image: alpine:latest
                command:
                  - sh
                  - -c
                  - |
                    set -eu
                    
                    apk --update add git
          
                    cd /workdir
                    echo "Start to Clone "{{workflow.parameters.repo_url}}
                    git -C "{{workflow.parameters.repo_name}}" pull || git clone {{workflow.parameters.repo_url}} 
                    cd {{workflow.parameters.repo_name}}
          
                    echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
                    git checkout {{workflow.parameters.target_branch}}
                    
                    echo "Get commit id" 
                    git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
                    commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
                    echo "Commit id is got: "$commitId
                                        
                    echo "Git Clone and Checkout Complete."
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
                resources:
                  requests:
                    memory: 1Gi
                    cpu: 1
                activeDeadlineSeconds: 1200
            depends: git-clone

提交Workflow参数说明

涉及参数如下：

说明

repo_url需要为ssh格式。

方法三：基于Git Clone命令与用户名密码

和前两种方法不同，该方法不需要增加DAG（Directed Acyclic Graph）任务，但需要修改git-checkout-pr中git clone的命令，并通过env引用git-creds secret中的用户名、密码。命令如下：

git clone https://${GIT_USER}:${GIT_TOKEN}@github.com/${GITHUB_REPOSITORY}

示例模板

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: ci-git
spec:
  entrypoint: main
  volumes:
  - name: run-test
    emptyDir: {}
  - name: workdir
    persistentVolumeClaim:
      claimName: pvc-nas
  - name: docker-config
    secret:
      secretName: docker-config
  arguments:
    parameters:
    - name: repo_url
      value: ""
    - name: repo_name
      value: ""
    - name: target_branch
      value: "main"
  templates:
    - name: main
      dag:
        tasks:
          - name: git-checkout-pr
            inline:
              container:
                image: alpine:latest
                env:
                - name: GIT_USER
                  valueFrom:
                    secretKeyRef:
                      name: git-creds
                      key: username
                - name: GIT_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: git-creds
                      key: password
                command:
                  - sh
                  - -c
                  - |
                    set -eu
                    
                    apk --update add git
          
                    cd /workdir
                    echo "Start to Clone "{{workflow.parameters.repo_url}}
                    git -C "{{workflow.parameters.repo_name}}" pull || git clone https://$GIT_USER:$GIT_TOKEN@{{workflow.parameters.repo_url}} 
                    cd {{workflow.parameters.repo_name}}
          
                    echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
                    git checkout {{workflow.parameters.target_branch}}
                    
                    echo "Get commit id" 
                    git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
                    commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
                    echo "Commit id is got: "$commitId
                                        
                    echo "Git Clone and Checkout Complete."
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
                resources:
                  requests:
                    memory: 1Gi
                    cpu: 1
                activeDeadlineSeconds: 1200

提交Workflow参数说明

涉及参数如下：

说明

repo_url参数不能包含https://前缀。

容器服务 Kubernetes 版 ACK：在CI Pipeline中Clone私有Git仓库

背景信息

在工作流集群中保存私有仓库凭据

方法一：基于Argo Workflows Git Artifact与用户名密码

示例模板

提交Workflow参数说明

方法二：基于Argo Workflows Git Artifact与SSH Private Key

示例模板

提交Workflow参数说明

方法三：基于Git Clone命令与用户名密码

示例模板

提交Workflow参数说明

相关文档