Hybrid CI/CD - Hosting Dockerized Jenkins Slaves On Alibaba Cloud

This article shares how to implement a distributed and dockerized Jenkins Slave Pool on Alibaba Cloud which enables dynamic and parallel CI/CD pipeline provisioning.

By John Hua, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This article shares how to implement a distributed and dockerized Jenkins Slave Pool on Alibaba Cloud which enables dynamic and parallel CI/CD pipeline provisioning to accomplish best resource utilization, on-demand scalability and cost-effectiveness.

What Is Hybrid CI/CD

Hybrid CI/CD is a distributed platform that uses a mix of on-premises and public cloud elastic compute resources for CI/CD purpose. Hybrid CI/CD basically control the workloads between private and public clouds to balance needs and costs.

At a high level, the architecture is shown in below diagram.

The Benefits Of Hybrid CI/CD

Scalability & Flexibility

On-premises do offer a certain level of scalability, which is still per data center and highly depends on their budget. And often times when one system consumes resources than expected, all other systems are affected. So how to set the boundary for budgeting becomes tricky. Public cloud services will offer greater scalability with larger cloud infrastructure. Considering CI/CD jobs are usually isolated, short lived and generally stateless. By moving CI/CD jobs to the public cloud, it allows an organization to reserve scalability for the core services.

Cost Effectiveness

Public clouds are more likely to offer more economic cost effectiveness as the cost of centralized management are shared by every user.

Security

Compared to pure public cloud solution, Hybrid CI/CD reserves the sensitiveness for security demand as you can always restrict data to be accessible only to applicable CI/CD jobs.

Why Docker Host As Build Slaves

Resource Utilization

Docker will dynamically assign resources so you can have multiple Jenkins Slaves running in parallel. It will tremendously increase the throughput of the CI/CD pipeline.

Isolation

Docker ensures your applications and resources are isolated and segregated. You can simply destroy the target container without worrying about if Jenkins Slave really clean up thoroughly.

Environment Standardization

You can simply build CI/CD environment inherited from the standard images.

Jenkins Slave Container Implementation

There are two types of communication protocols from Jenkins Master to Jenkins Slaves being used. They are Java Web Start (aka JNLP) and SSH. Although SSH is encrypted under Blocking I/O, which limits the scalability, but it is still the most common method because it is less dependent on the load balancer. So in this article we will only discuss the type of ssh slave.

It is recommended to build Jenkikns Slave docker images specifically as environment may vary project from project. https://github.com/jenkinsci/docker-ssh-slave has provided a great example to follow.

Dockerfile

    FROM openjdk:8-jdk
    LABEL MAINTAINER="Nicolas De Loof <nicolas.deloof@gmail.com>"

    ARG user=jenkins
    ARG group=jenkins
    ARG uid=1000
    ARG gid=1000
    ARG JENKINS_AGENT_HOME=/home/${user}

    ENV JENKINS_AGENT_HOME ${JENKINS_AGENT_HOME}

    RUN groupadd -g ${gid} ${group} \
        && useradd -d "${JENKINS_AGENT_HOME}" -u "${uid}" -g "${gid}" -m -s /bin/bash "${user}"

    # setup SSH server
    RUN apt-get update \
        && apt-get install --no-install-recommends -y openssh-server \
        && rm -rf /var/lib/apt/lists/*
    RUN sed -i /etc/ssh/sshd_config \
            -e 's/#PermitRootLogin.*/PermitRootLogin no/' \
            -e 's/#RSAAuthentication.*/RSAAuthentication yes/'  \
            -e 's/#PasswordAuthentication.*/PasswordAuthentication no/' \
            -e 's/#SyslogFacility.*/SyslogFacility AUTH/' \
            -e 's/#LogLevel.*/LogLevel INFO/' && \
        mkdir /var/run/sshd

    VOLUME "${JENKINS_AGENT_HOME}" "/tmp" "/run" "/var/run"
    WORKDIR "${JENKINS_AGENT_HOME}"

    COPY entrypoint.sh /usr/local/bin/entrypoint.sh

    EXPOSE 22

    ENTRYPOINT ["entrypoint.sh"]

Jenkins Slave entrypoint.sh

    #!/bin/bash -ex

    write_key() {
        mkdir -p "${JENKINS_AGENT_HOME}/.ssh"
        echo "$1" > "${JENKINS_AGENT_HOME}/.ssh/authorized_keys"
        chown -Rf jenkins:jenkins "${JENKINS_AGENT_HOME}/.ssh"
        chmod 0700 -R "${JENKINS_AGENT_HOME}/.ssh"
    }

    if [[ $JENKINS_SLAVE_SSH_PUBKEY == ssh-* ]]; then
    write_key "${JENKINS_SLAVE_SSH_PUBKEY}"
    fi
    if [[ $# -gt 0 ]]; then
    if [[ $1 == ssh-* ]]; then
        write_key "$1"
        shift 1
    else
        exec "$@"
    fi
    fi

    # ensure variables passed to docker container are also exposed to ssh sessions
    env | grep _ >> /etc/environment

    ssh-keygen -A
    exec /usr/sbin/sshd -D -e "${@}"

Jenkins Master Configuration

We assume you have already set up your Jenkins Master before. You may also refer to below trilogy articles to build a new one on Alibaba Cloud:

Then go to Jenkins Dashboard / Manage Jenkins / Manage Plugins, search for "Docker Plugin" and install it

Next, go to Jenkins Dashboard / Manage Jenkins / Configure system, under docker fill in Docker URL with your ECS public ip address and the port of docker remote API, which by default is 2375. There is a "test connection" button so you can try and see if the connection is successful.

After that choose "Add Docker Template" and click "docker template". And fill in the details according to below:

Docker Image: jenkinsci/ssh-slave
Remote Filing System Root: /home/jenkins
Labels: ssh-slave
Credentials: the public key you have injected for ssh-slave container

When everything is set, you can click "Save" button.

Test Configuration

Go to "New item" and create a freestyle project named "ssh-slave-test".

Under Restrict, type the label name you have given in the slave template. In this case, it is "ssh-slave".

Under Build select the execute shell option and type an echo statement as shown below.

    echo "hello {JOB_NAME}"

Save this job and click build now, you should get the following output.

    hello ssh-slave-test

What Is Next

Once you finished everything above, you can integrate with your code repository and create real world CI/CD jobs. You may also customize the docker image based on your actual requirement.

Community

Hybrid CI/CD - Hosting Dockerized Jenkins Slaves On Alibaba Cloud

What Is Hybrid CI/CD

The Benefits Of Hybrid CI/CD

Scalability & Flexibility

Cost Effectiveness

Security

Why Docker Host As Build Slaves

Resource Utilization

Isolation

Environment Standardization

Jenkins Slave Container Implementation

Jenkins Master Configuration

Test Configuration

What Is Next

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

Robotic Process Automation (RPA)

DevOps Solution

Alibaba Cloud Flow

ACK One