Accelerate AI image building - Container Service for Kubernetes

When you create a container by using an image, the system typically downloads and decompresses the whole package of the image even if only some resources are required to start the container. This usually takes a long period of time. You can use Container Registry Enterprise Edition to deploy an accelerated version of a container image. The accelerated version allows the system to download and decompress only the required resources without the need to download the entire image package. This accelerates application deployment and provides high elasticity. This topic uses a Stable Diffusion application as an example to describe how to use Container Registry Enterprise Edition to build AI images and accelerate image pulling.

Prerequisites

A Container Registry Enterprise Edition instance whose instance type is Standard or Advanced is created. For more information, see Create an instance of Container Registry Enterprise Edition.
The preparations for using Container Registry Enterprise Edition instances are completed, including binding a source code hosting platform, creating a namespace, and creating an image registry. For more information, see Steps 1 to 3 in Use Container Registry Enterprise Edition instances to build images.
An ACK Pro cluster is created. For more information, see Create an ACK managed cluster.
A kubectl client is connected to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Limits

Only the following regions support the on-demand image loading feature: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Heyuan), China (Chengdu), China (Hong Kong), Singapore, US (Silicon Valley), US (Virginia), and Japan (Tokyo).

Introduction to Stable Diffusion

Stable Diffusion WebUI is a popular project on Github that allows you to generate images from prompts. Stable Diffusion consists of a text understanding component and an image generating component. Stable Diffusion encodes prompts by using the CLIP model and generates images by using a diffusion model.

Important

Alibaba Cloud does not guarantee the legitimacy, security, or accuracy of the third-party model Stable Diffusion. Alibaba Cloud shall not be held liable for any damages caused by the use of Stable Diffusion.
You must abide by the user agreements, usage specifications, and relevant laws and regulations of Stable Diffusion. You shall bear all consequences resulting from the legitimacy and compliance requirements of Stable Diffusion.

Step 1: Create a Dockerfile on the source code hosting platform

To use a Container Registry Enterprise Edition instance to build images, you must first bind the instance to a source code hosting platform. Container Registry supports the following source code hosting platforms: Gitee, GitHub, and GitLab.

After you bind a source code hosting platform as described in Prerequisites, create a Dockerfile on the platform. The Dockerfile will be used to pull Stable Diffusion application images. The following code block is an example:

FROM yunqi-registry.cn-shanghai.cr.aliyuncs.com/lab/stable-diffusion:model-only-v1.5 as model

FROM docker.io/nvidia/cuda:12.1.0-cudnn8-runtime-ubuntu22.04

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
    libgl1 libglib2.0-0 wget git curl vim python3.10 python3-venv && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git && \
cd stable-diffusion-webui && git checkout v1.6.0

COPY --from=model /data/v1-5-pruned-emaonly.safetensors /stable-diffusion-webui/models/Stable-diffusion/
COPY --from=model /root/.cache/huggingface /root/.cache/huggingface

WORKDIR /stable-diffusion-webui/

RUN ./webui.sh -f can_run_as_root --exit --skip-torch-cuda-test

ENV VIRTUAL_ENV=/stable-diffusion-webui/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

CMD ["python3", "launch.py", "--listen --skip-torch-cuda-test --no-half"]

Step 2: Use Container Registry Enterprise Edition to accelerate Stable Diffusion application image building

In this section, a Container Registry Enterprise Edition instance that supports image acceleration is created. After image acceleration is enabled, images pushed to the image registry are automatically converted to accelerated images.

Log on to the Container Registry console.
In the top navigation bar, select a region.
Click the Container Registry Enterprise Edition instance that you created. In the left-side navigation pane, choose Repository > Repositories.

Click Create Repository, configure parameters in the dialog box that appears, and then click Create Repository.

The following table describes the key parameters. For more information, see Step 3: Create an image repository.

Parameter	Description and example	Layout of the console
Namespace	Select a namespace.
Repository Name	Enter the name of the image repository.
Accelerated Image	After you enable image acceleration, images pushed to the image repository are automatically converted to accelerated images. Image acceleration is enabled in this topic.
Summary	Enter a summary.
Code Source	Select the source code hosting platform that is bound to the Container Registry Enterprise Edition instance and contains a Dockerfile.
Build Settings	Select Automatically Build Images When Code Changes and Build With Servers Deployed Outside Chinese Mainland. Automatically Build Images When Code Changes: The building rule is automatically triggered when code is committed from a branch. Build With Servers Deployed Outside Chinese Mainland: Images are built on servers outside the Chinese mainland and then pushed to a repository in the specified region. If the Dockerfile used in your project must be downloaded from a site outside the Chinese mainland but the cross-border network connection is unstable, you can enable Build With Servers Deployed Outside Chinese Mainland.

Go to the details page of the image repository. In the left-side navigation pane, click Build. In the Build Rules section, click Add Build Rule. Configure the parameters in the wizard and click OK.

The following table describes the key parameters. For more information, see Step 4: Build an image.

Parameter	Description and example	Layout of the console
Required	Specify the type of the source code repository. Valid values: Branch and Tag.
Branch/Tag	Select or enter a branch or a tag. Regular expressions are supported. If you specify the release-(?<imageTag>\w) regular expression, the system automatically builds an image of V1 when the source code in the release-v1 branch is updated. You cannot manually build the image. For more information about how to specify regular expressions, see Use regular expressions in named capturing groups. Note* After you specify regular expressions, images can be built only by the system. You cannot manually build images.
Build Context Directory	Specify the directory in which the Dockerfile resides. This is the subdirectory of the branch or tag directory. For example, if the branch directory is master/ and the Dockerfile is in the master/ directory, the value is master/Dockerfile.
Dockerfile Filename	Specify the name of the Dockerfile. The default name is Dockerfile.
Image Tag	Specify the image tag, such as 1.0.0. You can enable named capturing groups. For example, if you specify a named capturing group for Branch/Tag, you can use the captured content.
Build Configurations	Configure the build architecture and build parameters. In this topic, the default settings are used.

In the Build Rules section of the Build Settings page, click Build in the Actions column of the build rule.
After you complete the configuration, you can view the status of the image building task in the Build Log section. If the status is Success, the image is built.
When image acceleration is enabled, it requires about 30 minutes to build the image.
After the image is built, click Image Tag in the left-side navigation pane and view the tag of the image whose type is Accelerated Image and the tag of the image whose type is Container Image.
The current repository is an accelerated image repository. All images in the repository are automatically converted to accelerated images. In the preceding figure, two accelerated images and one regular container image are created.
In the section of the image whose type is Container Image, click Tag and copy Repository URL. The URL will be used by the Stable Diffusion application in the ACK cluster.

Step 3: Accelerate the Stable Diffusion application in the ACK cluster

1. Install the image acceleration component

To enable container acceleration, you must install the image acceleration component on worker nodes.

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose Operations > Add-ons in the left-side navigation pane.
On the Add-ons page, find and click aliyun-acr-acceleration-suite and follow the instructions to install the component.

2. Create a node pool that uses the accelerated image

To test the accelerated image, you must create a node pool. When you create the node pool, add the alibabacloud.com/image-accelerate-enabled: true label to nodes to use image acceleration during node initialization.

Note

The alibabacloud.com/image-accelerate-enabled: true label does not take effect on existing nodes. We recommend that you create a separate node pool to manage nodes that use image acceleration.

On the Clusters page, click the name of the cluster that you want to manage and choose Nodes > Node Pools in the left-side navigation pane.
In the upper-right corner of the page, click Create Node Pool. Create a CPU-accelerated node by setting Expected Nodes to 1. Select an instance type of 16 vCores and 32 GB of memory or higher. Then, click Advanced Options and set Node Label to alibabacloud.com/image-accelerate-enabled: true.
For more information, see Create and scale out a node pool.

Run the following command to query nodes that have the image acceleration label:

kubectl get nodes -l alibabacloud.com/image-accelerate-enabled=true

Expected output:

NAME                       STATUS   ROLES    AGE     VERSION
cn-hangzhou.172.XX.XX.115   Ready    <none>   2m33s   v1.24.6-aliyun.1

3. Configure a VPC for the image repository

To ensure that the ACK cluster can pull images from the image repository, you must specify the virtual private cloud (VPC) of the ACK cluster for the image repository in the Container Registry console.

Log on to the Container Registry console.
In the top navigation bar, select a region.
Click the Container Registry Enterprise Edition instance. In the left-side navigation pane, choose Repository > Access Control.
Click Add VPC, select the VPC of the ACK cluster and the vSwitch of the nodes that have the image acceleration label, and then click OK.
After the status of the VPC changes to Running, the VPC is added.
Note
To view the information about the VPC and the vSwitch, log on to the ACK console. On the Clusters page, click the name of the ACK cluster and perform the following steps:
- View the VPC of the ACK cluster: In the left-side navigation pane, click Cluster Information. Then, click the Cluster Resources tab to view the VPC information.
- View the vSwitch of the nodes that have the image acceleration label: In the left-side navigation pane, choose Nodes > Node Pools. Click the name of the node pool that you want to view and then click the Overview tab. You can view the vSwitch information in the Node Configurations section.

4. Deploy application pods in the ACK Pro cluster

You can use a YAML file to pull the accelerated image from the Container Registry instance and deploy the image in the ACK cluster.

Run the following command to create a namespace for the Stable Diffusion application:
```
kubectl create ns accelerate-ai-demo
```

Run the following command to add the image acceleration label to the namespace:

kubectl label namespaces accelerate-ai-demo k8s.aliyun.com/image-accelerate-mode=on-demand

Run the following command to create acr-credential-test in the accelerate-ai-demo namespace to store the credentials that are used to pull images.
Replace <yourInstanceName>-<registry-vpc.cn-hangzhou.cr.aliyuncs.com> with the repository URL, <yourUserName> with the username of your Alibaba Cloud account, and <yourPassword> with the password of the Container Registry instance.
```
kubectl -n accelerate-ai-demo create secret docker-registry acr-credential-test --docker-server=<yourInstanceName>-<registry-vpc.cn-hangzhou.cr.aliyuncs.com> --docker-username=<yourUserName> --docker-password=<yourPassword>
```

Use the following code to create a file named stable-diffusion-ack-gpu-accelerate.yaml. The file is used to deploy a Deployment named tea and a Service named tea-svc for the Stable Diffusion application.

View sample code

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: stable-diffusion
  name: stable-diffusion
  namespace: accelerate-ai-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: stable-diffusion
  template:
    metadata:
      labels:
        app: stable-diffusion
    spec:
      containers:
      - args:
        - --listen
        - --skip-torch-cuda-test 
        - --no-half
        command:
        - python3
        - launch.py
        image: <Stable Diffusion application image name> # The URL of the repository from which the Stable Diffusion application image is pulled. 
        imagePullPolicy: Always
        name: stable-diffusion
        ports:
        - containerPort: 7860
          protocol: TCP
        readinessProbe:
          tcpSocket:
            port: 7860
        resources:
          requests:
            cpu: "8"
            memory: 16Gi
            ephemeral-storage: 50Gi
      imagePullSecrets:
        - name: acr-credential-test
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: internet
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: PayByCLCU
  name: stable-diffusion
  namespace: accelerate-ai-demo
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 7860
    protocol: TCP
    targetPort: 7860
  selector:
    app: stable-diffusion
  type: LoadBalancer

Run the following command to deploy the Stable Diffusion application:
```
kubectl apply -f stable-diffusion-ack-gpu-accelerate.yaml
```

Run the following command to query the status of the pod deployed for the Stable Diffusion application:

kubectl -n accelerate-ai-demo get pod

Expected output:

NAME                                READY   STATUS    RESTARTS   AGE
stable-diffusion-75759ff59f-hw5zm   1/1     Running   0          73s

Run the following command to query the details of the pod:

kubectl -n accelerate-ai-demo describe pod <pod-name>

Expected output:

...
Events:
  Type    Reason          Age   From               Message
  ----    ------          ----  ----               -------
  Normal  Scheduled       23s   default-scheduler  Successfully assigned accelerate-ai-demo/stable-diffusion-75ff55794f-5mj8p to cn-hangzhou.172.16.7.99
  Normal  AllocIPSucceed  19s   terway-daemon      Alloc IP 172.16.7.115/24
  Normal  Pulling         19s   kubelet            Pulling image "XXX"
  Normal  Pulled          17s   kubelet            Successfully pulled image "XXX" in 1.61s (1.61s including waiting)
  Normal  Created         17s   kubelet            Created container stable-diffusion
  Normal  Started         16s   kubelet            Started container stable-diffusion

The output indicates that the Stable Diffusion application is ready after about 7 seconds. The actual time depends on your environment.

When image acceleration is disabled, the ACK cluster requires about 4 minutes to pull the image. The actual time depends on your environment. Image acceleration greatly reduces the image pulling time of ACK and accelerates the launch of the Stable Diffusion application.

Step 5: Verify the configuration

Run the following command to query the Service details of the pod deployed for the Stable Diffusion application. Record the external IP address and use it to access the application.
```
kubectl -n accelerate-ai-demo get svc 
```
Expected output:
```
NAME               TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
stable-diffusion   LoadBalancer   172.XX.XX.163   121.XX.XX.64   7860:32621/TCP   116s
```
Record the value of EXTERNAL-IP.
Run the following curl command to test the Stable Diffusion application:
Replace <EXTERNAL-IP> with the value of EXTERNAL-IP that you recorded.
```
curl http://<EXTERNAL-IP>:7860/ 
```
You can also access http://<EXTERNAL-IP>:7860/ in a browser. In the upper-left corner, select a checkpoint. In the Prompt section, enter prompts and click Generate to generate an image.
A delay may exist during the first time you access the application. Wait a period of time or refresh the page.