Platform For AI:Deploy a model service by using a custom image

Prepare a custom image

Develop an image service

You can develop an image service by using multiple methods. For example, you can use Flask to start a web server for service access. The following sample code shows a simple service whose code file is app.py.

from flask import Flask

app = Flask(__name__)

@app.route('/hello/model')
def hello_world():
    return 'Hello World'


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000)

Build an image

The following section describes how to install Docker to build an image on your on-premises machine. You can also use Alibaba Cloud Container Registry to build an image. For more information about how to create a container image, see What is Container Registry?

1. Write a Dockerfile.

   A Dockerfile is a text document that contains the instructions and descriptions required to build an image. You need to create a Dockerfile at the same level as the app.py file. The Dockerfile has the following directory structure:

   flask_app
     ├── app.py
     └── Dockerfile    

   In the following Dockerfile, the app.py model service file is packaged in the image that you want to build. You can also save the model service file in Object Storage Service (OSS), File Storage NAS (NAS), or a Git repository and mount the model service file to service instances during the runtime. For more information, see Mount storage to services.

   # Specify the desired image.
   FROM alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/python
   # Run the following commands in the container:
   RUN yum -y install python3-pip
   RUN pip3 install flask 
   # Create a folder named app.
   RUN mkdir -p /app
   # Go to the app directory.
   RUN cd /app
   # Copy the app.py file in the current directory of the Linux system to the /app directory of the container. 
   COPY app.py /app/
   # Expose port 8000.
   EXPOSE 8000
   # Set the app folder as a working directory.
   WORKDIR /app
   
   # Run the following command when the container is started:
   CMD ["flask", "run", "--host", "0.0.0.0"]

2. Build an image on your on-premises machine. In the CLI, go to the flask_app directory and run the following command to build a Docker image named flask-app:

   docker build -t flask-app . 

Push an image to an image repository

EAS pulls the image from an image repository for deployment. The following section describes how to push an image that you built on your on-premises machine to an image repository:

1. Run the docker images command to obtain the ID of the image that you built.

2. Select an image repository. In this example, the Container Registry image repository is used. The repository address is crpi-***.cn-hangzhou.personal.cr.aliyuncs.com/***/***. Different image repositories have different requirements for network configurations during deployment in EAS. For more information, see Select an image repository.

3. Run the following commands to push the image to the Container Registry image repository:

   docker login --username=<Username>  crpi-***.cn-hangzhou.personal.cr.aliyuncs.com
   docker tag <Image ID> crpi-***.cn-hangzhou.personal.cr.aliyuncs.com/***/***:flask-app
   docker push crpi-***.cn-hangzhou.personal.cr.aliyuncs.com/***/***:flask-app

   The image address is crpi-***.cn-hangzhou.personal.cr.aliyuncs.com/***/***:flask-app.

Select an image repository

EAS requires different network configurations for different image repositories.

• Container Registry repository:

  • Container Registry Personal Edition: provides a unified internal endpoint for each region. If the model service that you want to deploy resides in the region in which a Container Registry Personal Edition image resides, you can use the internal endpoint of the image to deploy the model service. Otherwise, you must use the public endpoint of the image to deploy the model service.

  • Container Registry Enterprise Edition: A Container Registry Enterprise Edition image can be accessed only within your virtual private cloud (VPC). To allow EAS to pull the image, you must connect EAS to the corresponding VPC. For more information, see Configure VPC.

• Self-built image repository:

  • If you use Harbor to create a self-managed image repository in a VPC, the image repository is accessible only within your VPC. Similar to using a Container Registry Enterprise Edition image, you must connect EAS to your VPC. For more information, see Configure VPC.

Authenticate access to an image repository

• If you use a Container Registry image repository under the same account for model service deployment, you do not need to enter the username and password. EAS allows you to pull an image across services without using a password.

• If your self-built image repository requires a username and a password as access credentials, you must configure the dockerAuth parameter when you deploy a model service. For more information, see Deploy a model service.

Deploy a model service

Obtain the required information to deploy a model service by using a custom image:

• The address of the image repository. Example: crpi-***-vpc.cn-hangzhou.personal.cr.aliyuncs.com/***:flask-app.

• The command to start a container based on the image. Example: python3 app.py.

• The port number of the container that the service listens on. Example: 8000.

  Note

  The port number is optional. For example, if you want the service to accept requests from message queues within the container instead of the EAS gateway, you do not need to specify the port number.

Appendix: Sample service configurations

Use the gRPC protocol

{
  "name": "image_test",
  "containers": [
    {
      "image": "registry-vpc.cn-shanghai.aliyuncs.com/xxx/yyy:zzz",
      "command": "python app.py",
      "port": 8000
    }
  ],
  "metadata": {
    "instance": 1,
    "enable_grpc": true
  }
}

References

• The health check feature is supported when you deploy model services by using custom images. For more information, see Configure the health check feature.

• For information about how deployment works in EAS, see Service deployment.

• After you deploy a model service, you can use the auto scaling feature to handle workload fluctuations in your business. For more information, see Auto scaling.