How to Develop Function Compute - TensorFlow Serving

This tutorial is a part of the 'How to Develop Function Compute' series. It uses an example to demonstrate how to use Fun and NAS for TensorFlow serving.

By Du Wan (Yixian)

Preface

First, let's take a quick look at several key concepts mentioned in this article:

Function Compute: It's an event-driven service that allows users to focus on writing and uploading code without the need to manage infrastructure such as servers. Function Compute provides compute resources, allowing users to run code more elastically by just paying for the resources consumed while running the code. For more information about Function Compute, see here.
Fun: Also known as Funcraft, is a developer tool for serverless applications that helps in managing resources such as Function Compute, API Gateway, and Log Service. Use Fun to develop, build, and deploy resources by defining specified resources in the template.yml file. For more information about Fun, see here.

Note: The operations in this article are applicable to Fun version 2.16.0 and and all later versions.

Dependent Tools

This project is developed based on macOS. However, the tools involved are platform-independent and also applicable to the Linux and Windows operating systems. Before proceeding with the example, make sure that the following tools are correctly installed, updated to the latest version, and properly configured.

Docker
Fun

Fun is based on Docker to simulate the local environment.

MacOS users may use homebrew to install these tools:

brew tap vangie/formula
brew install fun
brew install fcli

Windows and Linux users must read the following articles to install these tools:

After the installation, first run the fun config command to initialize the configuration.

Note: Make sure you are using Fun version 2.16.0 or any later version.

$ fun --version
2.16.0

Background

AI model serving is a typical scenario of Function Compute. Data engineers train a model and require software engineers to convert the model into a system or service. This process is called model serving. Function Compute features O&M-free and auto-scaling, meeting the requirement of model serving for a high-availability distributed system. This article introduces an example of deploying a model trained by TensorFlow CharRNN for automatically writing Chinese ancient poems in five-character quatrain style to Function Compute. The Python TensorFlow dependency library and training model files constitute hundreds of MB, which exceeds the upper limit of the code package size (50 MB) in Function Compute. For such ultra-large files, the NAS file system is the best choice. This article introduces how to use Fun and NAS for TensorFlow serving.

Quick Start

1) Clone the Poetry Project

git clone https://github.com/vangie/poetry.git

2) Install Dependencies

Run the fun install command to install the dependencies.

The fun.yml file declares TensorFlow dependencies and contains the script commands for training models. Therefore, it takes more time to run the command. Note that the target property is specified in the fun.yml file. Therefore, the dependencies are installed in the locally simulated NAS Directory.

The following figure shows the dynamic demonstration. To obtain a better demonstration effect, set the max_steps parameter to 100.

3) Call the Function

Run the fun local invoke command to call the function locally. The following figure shows the successful output.

$ fun local invoke poetry                                             
Reading event data from stdin, which can be ended with Enter then Ctrl+D
(you can also pass it from file with -e)
mouting local nas mock dir /Users/vangie/Desktop/poetry/.fun/nas/3be7b4835d-pvs14.cn-shanghai.nas.aliyuncs.com/ into container /mnt/nas

skip pulling image aliyunfc/runtime-python3.6:1.5.2...
FunctionCompute python3 runtime inited.
FC Invoke Start RequestId: 938334c4-5407-4a72-93e1-6d59e52774d8
.......£¨Ê¡ÂÔÁË²¿·ÖÈÕÖ¾£©
²»¼û½ÖÐ¿Í£¬ÎÞÑÔ´Ë±ð¹é¡£
½·çÇïÓêÂä£¬É½É«Ò¹É½³¤¡£
²»ÎÊ½ÄÏ¿Í£¬¹ÂÖÛÔÚ¹ÊÏç¡£
Ò»ÄêÈçÔ¶±ð£¬ºÎ´¦ÊÇ¹éÈË¡£
Ò»Ò¹ÎÞÈË

RequestId: 938334c4-5407-4a72-93e1-6d59e52774d8          Billed Duration: 14074 ms       Memory Size: 1998 MB    Max Memory Used: 226 MB

The following figure shows the dynamic demonstration.

4) Upload the Local Machine Learning Dependencies and Models to NAS

In the previous step, we ran the fun local invoke command to call the Fun NAS function locally. However, these NAS resources are stored in the local system. Therefore, we still need to upload them to the NAS service.

The following sections describe two uploading methods: Fun NAS and ECS.

4.1 Uploading Through Fun NAS (Recommended)

1) Run the fun nas init command to initialize NAS configurations.
2) Run the fun nas info command to view the local directory of NAS. Skip this step as the fun install target command to directly specify the location is already executed in the previous step.
3) Run the fun nas sync command to upload local NAS resources to the NAS service in the cloud.
4) Run the fun nas ls nas://poetry:/mnt/auto/ command to check whether the local files have been uploaded to the NAS service.

The following figure shows the dynamic demonstration of running the Fun-NAS sync command. Here, we reduce the dependency size for better demonstration effect.

4.2 Uploading Through ECS

Alternatively, upload local NAS resources through an Elastic Compute Service (ECS) instance.

Run the following commands to compress the local files, copy them to the NAS directory, and decompress them, as shown in the following figure.

# ¹ÒÔØ nas ÍøÅÌ
mount -t nfs -o vers=4.0 3be7b4835d-pvs14.cn-shanghai.nas.aliyuncs.com:/ /mnt/nas

# Ñ¹Ëõ±¾µØÒªÉÏ´«µÄÄ¿Â¼
cd .fun/nas/3be7b4835d-pvs14.cn-shanghai.nas.aliyuncs.com/
tar -czvf nas.tar.gz lib model

# ¿½±´µ½ nas Ä¿Â¼
scp nas.tar.gz root@47.103.83.174:/mnt/nas

# ½âÑ¹
tar -xvf nas.tar.gz

5) Deploy the Function

Run the fun deploy command to deploy the function to the remote end.

$ fun deploy             
using region: cn-shanghai
using accountId: ***********4733
using accessKeyId: ***********EUz3
using timeout: 60

Waiting for service poetry to be deployed...
        Waiting for function poetry to be deployed...
                Waiting for packaging function poetry code...
                package function poetry code done
        function poetry deploy success
service poetry deploy success

Use fcli to call the function from the remote end. Alternatively, call the function in the Function Compute console.

$ fcli function invoke -s poetry -f poetry

Now, observe that the ancient poetry creation program has been deployed to Function Compute.

Community

How to Develop Function Compute - TensorFlow Serving

Preface

Dependent Tools

Background

Quick Start

1) Clone the Poetry Project

2) Install Dependencies

3) Call the Function

4) Upload the Local Machine Learning Dependencies and Models to NAS

4.1 Uploading Through Fun NAS (Recommended)

4.2 Uploading Through ECS

5) Deploy the Function

Read previous post:

Read next post:

Alibaba Cloud Serverless

You may also like

Comments

Alibaba Cloud Serverless

Related Products

Function Compute

Serverless Workflow

Serverless Application Engine

Realtime Compute for Apache Flink