Community

By Rick Yang

Nowadays, DevOps, IaC, and CI/CD have been discussed and adopted in different ways. It's difficult to tell which one is better than the other. However, only you and your team know which one is suitable for your specific situation. With that said, as a Solutions Architect from Alibaba Cloud, I decided to put my feet into my customer's shoes and made a CI/CD demo with open-source and Alibaba Cloud managed services.

Again, it may not be the best practice, but please treat it as a starting point for further discussion. Let's get started with the following objectives.

Objectives:

Create a demo web application with the FastAPI module and test it locally by building and running a Docker image of it
Create a project on GitLab and host the source code in the code repository
With Terraform, create an ASK cluster or Alibaba Serverless Kubernetes as the web application deployment environment
Create a CI/CD pipeline consisting of three main stages: Test, Build/Push, Deploy
1. The unit test of the code can be run with GitLab runner
2. The ACR or Alibaba Container Registry is used to store the Docker image built and pushed by GitLab runner
3. The web application consisting of Deployment, Service, and Ingress can be deployed on the ASK cluster with kubectl and the yaml files.
With this setup, once the code update is committed on GitLab, the CI/CD pipeline will be triggered automatically.

Create a Containerized Web Application Locally

This web application is a demo API endpoint that can be accessed from a local host, as illustrated below:

Build the Docker image of this web application from scratch on your PC/Mac/Linux host:

Make a folder: mkdir fastapi_app
Package the requirements. For this demo, we need to create a requirements.txt file with the following content:

fastapi>=0.68.0,<0.69.0
pydantic>=1.8.0,<2.0.0
uvicorn>=0.15.0,<0.16.0

Create the new folder under fastapi_app and name it app
Create two files under the app folder: an empty file __init__.py and an main.py with the following content:

from typing import Union

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
return {"item_id": item_id, "q": q}

Create a Dockerfile with following contents, which is used to build the Docker image later

# 
FROM python:3.9

# 
WORKDIR /code

# 
COPY ./requirements.txt /code/requirements.txt

# 
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

# 
COPY ./app /code/app

# 
CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"]

Check the folder and file structure like this

.
├── app
│   ├── __init__.py
│   └── main.py
├── Dockerfile
└── requirements.txt

Go to the folder containing the requirements.txt file and execute the following commands (make sure you have installed Docker on your host. If not, please check this link)

docker build -t myimage
docker run -d --name mycontainer -p 80:80 myimage

Test it with access to the API endpoint http://127.0.0.1

Now, we can make sure the web application can be built and run locally without issues. Next, let's upload our code to the GitLab repository.

Create a GitLab Project and Store the Code

There are two ways to get this done, signing up for a gitlab.com account or creating a self-managed gitlab.yourdomain.com. I chose the second one, but either way is fine for this demo.

Create the GitLab project and upload the code files to it:

Sign in to your GitLab account and create a new project

Go into the project you just created and upload the code files that have been tested locally beforehand. Make sure the folder structure is the same as your local one:

Before creating the CI/CD pipeline on GitLab, we need to set up a k8s cluster for the web application deployment. Alibaba Serverless Kubernetes or ASK is used for this demo because I am from Alibaba Cloud :smile:, and it has the following features to help DevOps teams easily plan, test, and deploy their applications in an on-demand, fast, reliable, cost-effective, and secured way. Please click this link for more details about ASK

Create an ASK Cluster for Application Deployment with Terraform

Terraform is a well-known Infrastructure as Code (IaC) tool developed by HashiCorp.

With Providers and Modules, Terraform is pre-integrated with different IT vendors and cloud service providers, including Alibaba Cloud. Please click here for the latest documentation.

Create the ASK with Terraform Files Locally

The terraform-cli need to be installed on your local host as a precondition to the following steps. You should have an Alibaba Cloud RAM account with ACCESS_KEY and SECRET_KEY to access the Alibaba Cloud backend from Terraform.

Please check this link for more information

Make a folder ask_demo to host the Terraform files
Create three files to store your: main.tf, terraform.tfvars, and variables.tf main.tf contents:

terraform {
  required_providers {
    alicloud = {
      source = "aliyun/alicloud"
      version = "1.174.0"
    }
  }
}

//1) setup of alicloud RAM account with resource permisions and obtain the access_key and access_key_secret 
provider "alicloud" {
  access_key = var.my_access_key
  secret_key = var.my_secret_key
  region     = var.my_region
}

data "alicloud_zones" "abc_zones" {}

//2) setup of VPC
resource "alicloud_vpc" "test_vpc" {
  vpc_name   = var.my_vpc
  cidr_block = var.my_vpc_cidr_block
}

//3) setup of vswitch within the VPC
resource "alicloud_vswitch" "test_vswitch" {
  vswitch_name = var.my_vswitch
  vpc_id       = alicloud_vpc.test_vpc.id
  cidr_block   = var.my_vswitch_cidr_block
  zone_id      = data.alicloud_zones.abc_zones.zones.0.id
}
resource "alicloud_vswitch" "test_vswitch_bak" {
  vswitch_name = var.my_vswitch_bak
  vpc_id       = alicloud_vpc.test_vpc.id
  cidr_block   = var.my_vswitch_cidr_block_bak
  zone_id      = data.alicloud_zones.abc_zones.zones.2.id
}

resource "alicloud_cs_serverless_kubernetes" "serverless" {
  name                           = var.my_ask_name
  version                        = "1.22.3-aliyun.1"
  vpc_id                         = alicloud_vpc.test_vpc.id
  vswitch_ids                    = [alicloud_vswitch.test_vswitch.id,alicloud_vswitch.test_vswitch_bak.id]
  new_nat_gateway                = true
  endpoint_public_access_enabled = true
  deletion_protection            = false
  load_balancer_spec             = "slb.s2.small"
  time_zone                      = "Asia/Shanghai" 
  service_cidr                   = "172.21.0.0/20"
  service_discovery_types        = ["PrivateZone"]
}

terraform.tfvars contents

Replace "YOUR_ACCESS_KEY" and "YOUR_SECRET_KEY" with your RAM account information:

my_access_key  = "YOUR_ACCESS_KEY"
my_secret_key  = "YOUR_SECRET_KEY"

variables.tf contents

variable "my_access_key" {
  description = "RAM user access_key"
  sensitive   = true
}

variable "my_secret_key" {
  description = "RAM user secret_key"
  sensitive   = true
}

variable "my_region" {
  default = "cn-hongkong"
}

variable "my_vpc" {
  default = "test_vpc"
}

variable "my_vpc_cidr_block" {
  default = "10.1.0.0/21"
}

variable "my_vswitch" {
  default = "test_vswitch"
}
variable "my_vswitch_bak" {
  default = "test_vswitch_bak"
}

variable "my_vswitch_cidr_block" {
  default = "10.1.0.0/24"
}
variable "my_vswitch_cidr_block_bak" {
  default = "10.1.1.0/24"
}

variable "my_ask_name" {
  default = "ask001"
}

Under the same folder storing the files above, open your terminal and execute the following commands to plan and deploy the ASK k8s cluster on Alibaba Cloud
terraform init: If no error has been thrown out, you should see something like this:

Use the terraform plan command to prepare all the resources to be deployed on Alibaba Cloud without really creating them at this stage:

Use the terraform apply command with your confirmation to deploy the planned resources on Alibaba Cloud. It may take a few minutes to create the ASK cluster for this case.

Sign in to the Alibaba Cloud console and navigate to the ASK page to confirm whether the ASK cluster is created successfully:

Until now, we have GitLab for source code management and CI/CD pipeline. We also created the ASK k8s cluster for the application deployment, but how about the Docker image repository for image PUSH and PULL? We should have a place to store and manage the Docker image. This is where ACR comes into play.

Create the ACR or Alibaba Container Registry

Compared to the previous steps, creating the ACR is easy.

Sign in to the Alibaba Cloud console, navigate to the ACR page, and create your private repository.

You can find the login and pull and push instructions within the detail information page of the repository you just created, which will be used in your CI/CD pipeline configuration:

We need to have the yaml files for Deployment, Service, and Ingress creation to use the kubectl CLI tool to deploy the k8s based application. There are two ways to get there. You can create the files from scratch or copy them from the ASK GUI Console. I prefer the second option, which can save a lot of effort.

Generate the yaml Files with ASK GUI Console

To generate the yaml files of Deployment, Service, and Ingress, we need to create them manually with the ASK GUI Console:

Sign in to the Alibaba Cloud Console and navigate to the ASK console
Select Deployments and Create from image

Enter the name and the number of replicas (pods) and click Next:

Input the name and tag of the image, which can be copied from the ACR page in the previous step. We also need to specify the username and password we created for our ACR by clicking the Set Image Pull Secret and inputting the credentials as requested.

We should see the pod is up and running with the Docker image pulling from ACR:

Then, we can create the Service, which is used for internal communications between pods.

Create the Ingress, which is used to access the pods externally:

Test it with Ingress domain information:

Copy the yaml file information from the 'Deployment', 'Service', and 'Ingress' pages:

After the yaml files are ready, we can complete the last step of this demo, the CI/CD pipeline.

Create the CI/CD Pipeline on GitLab

Let's go back to the GitLab page and navigate to the project we created beforehand. Add a new file and name it .gitlab-ci.yml, which is used for editing the CI/CD scripts.

Click Edit and input the following contents with three stages: test, build, and deploy. As the names suggest:

The test stage is running the unit test script (Please find the test.py in the appendix section).

The build stage is to build the Docker image from the code and push it to the ACR.

The $ACR_USER and $ACR_PASSWORD represent the username and password created in ACR for image pushing authentication. They can be configured in the Settings—CI/CD—Variables.

The deploy stage is – First, login into a jumper host or local host and execute the kubectl to deploy the web application to the ACK k8s cluster. Assume the jumper host or local host has been installed with the kubectl CLI tool and configured with the $HOMEDIR/.kube/config file with ACK's connection information:

      stages:
    - test
    - build
    - deploy

run_test:
    stage: test
    image: python:3.9-slim-buster
    script:
        - python3 test.py

build_image:
    stage: build
    image: docker:20.10.16
    services:
        - docker:20.10.16-dind
    variables:
        DOCKER_TLS_CERTDIR: "/certs"
    script:
        - docker login -u $ACR_USER -p $ACR_PASSWORD registry-intl.cn-hongkong.aliyuncs.com
        - docker build -t registry-intl.cn-hongkong.aliyuncs.com/rickyang_namespace/rickyang_repo:myapp-1.0 .
        - docker push registry-intl.cn-hongkong.aliyuncs.com/rickyang_namespace/rickyang_repo:myapp-1.0

deploy:
    stage: deploy
    script:
        - chmod 400 $SSH_KEY
        - ssh -o StrictHostKeyChecking=no -i $SSH_KEY root@8.219.190.199 "
            ./test.sh &&
            kubectl create -f testapp.yml && 
            kubectl create -f testapp-svc.yml &&
            kubectl create -f testapp-ingress.yml &&
            touch mark.txt"

Finally, we can test our CI/CD pipeline. We need to update our code and click Commit Changes to trigger the pipeline. Then, GitLab, ACK, and ACR will take care of the rest.

Please remember to stop and remove the resources on ASK to save costs. It's super easy to do that with Terraform!

Remove the ASK Setup with Terraform

Open the terminal where you have your terraform-cli installed and go to the folder storing your three terraform files. Execute terraform destroy with confirmation:

Summary

Alright, we have completed a demonstration of a CI/CD pipeline consisting of Terraform, GitLab, ACR, and ACK. You must have preferences on each step in real-life scenarios. Then, try it and improve it to adapt to the dynamic business needs.

It is a one-click deployment solution, even though we took a lot of effort to make it happen. If this task can be repeated hundreds of times, isn't it worthy?

Appendix

The test.py file is a mock file used to simulate the unit test.

import unittest


class TestSum(unittest.TestCase):

    def test_sum(self):
        self.assertEqual(sum([1, 2, 3]), 6, "Should be 6")

    def test_sum_tuple(self):
        self.assertEqual(sum((1, 2, 3)), 6, "Should be 6")

if __name__ == '__main__':
    unittest.main()

The test.sh file in the jumper host is used to detect if the application has been deployed. If yes, the existing one will be deleted and re-deployed again.

#!/bin/bash
if [ -f "mark.txt" ];
then
 kubectl delete -f testapp-ingress.yml
 kubectl delete -f testapp-svc.yml
 kubectl delete -f testapp.yml
 rm mark.txt
else
 echo "The testapp does not exist yet"
fi

Community

CI/CD Pipeline Demo with Terraform, GitLab, ASK, and ACR

Create a Containerized Web Application Locally

Create a GitLab Project and Store the Code

Create an ASK Cluster for Application Deployment with Terraform

Create the ASK with Terraform Files Locally

Create the ACR or Alibaba Container Registry

Generate the yaml Files with ASK GUI Console

Create the CI/CD Pipeline on GitLab

Remove the ASK Setup with Terraform

Summary

Appendix

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

DevOps Solution

Alibaba Cloud Flow

Function Compute

Elastic High Performance Computing Solution