Use EASCMD or DSW to deploy services - Platform For AI - Alibaba Cloud Documentation Center

After you obtain a trained model, you can use a CLI to deploy the model as an online service in the Elastic Algorithm Service (EAS) module of Platform for AI (PAI). This topic describes how to use a CLI to deploy a model service.

Background information

EAS provides the following methods for CLI deployment:

Use EASCMD
The EASCMD client allows you to create, view, update, and delete model services on your server.
Use DSW
The Data Science Workshop (DSW) module of PAI integrates the EASCMD client. You can deploy models in DSW after model training is complete.

Prerequisites

Before you perform the operations that are described in this topic, make sure that the following requirements are met:

A trained model is obtained.
An AccessKey ID and an AccessKey secret are obtained. For more information about how to obtain the AccessKey ID and the AccessKey secret, see Create an AccessKey pair.
If you use DSW for deployment, a DSW instance must be created. For more information, see Create and manage DSW instances.

Use EASCMD

The EASCMD client allows you to create, view, update, and delete model services on your server. You can specify multiple instance types when you create a service. For more information, see Specify multiple instance types. The following steps show how to use the 64-bit Windows version of the EASCMD client to deploy a service.

Download the EASCMD client and complete identity authentication. For more information, see Download the EASCMD client and complete identity authentication.

Deploy a service.

Create a JSON file named test.json. Sample code:

{
    "metadata": {
        // If you use a dedicated resource group to deploy the service, configure the resource parameter. 
        "resource": "eas-r-9lkbl2jvdm0puv****",
        "instance": 1,
        "gpu": 0,
        "cpu": 1,
        "memory": 2000
    },
    // If you use the public resource group to deploy the service, configure the cloud.computing parameter to specify an instance type. 
    /*"cloud": {
        "computing": {
          "instance_type":"ecs.gn6i-c24g1.6xlarge"
        }
    },*/
    "name": "test_eascmd",
    "processor": "tensorflow_cpu_1.12",
    "model_path": "oss://cri-ldbuutu1acd51ryd-registry/linshi/pslr/"
}

Parameters:

model_path: Set this parameter to the path of the model file in your Object Storage Service (OSS) bucket. For information about other parameters in the file, see Parameters of model services.
cloud.computing: If you use the public resource group to deploy the service, set this parameter to the instance type that you want to use.

Run the following command in the directory that contains the JSON file to deploy the service:

eascmdwin64.exe create <test.json>

Replace <test.json> with the name of the JSON file that you created.

Sample success output:

[RequestId]: 1651567F-8F8D-4A2B-933D-F8D3E2DD****
+-------------------+----------------------------------------------------------------------------+
| Intranet Endpoint | http://166233998075****.cn-shanghai.pai-eas.aliyuncs.com/api/predict/test_eascmd |
|             Token | YjhjOWQ2ZjNkYzdiYjEzMDZjOGEyNGY5MDIxMzczZWUzNGEyMzhi****                   |
+-------------------+--------------------------------------------------------------------------+
[OK] Creating api gateway
[OK] Building image [registry-vpc.cn-shanghai.aliyuncs.com/eas/test_eascmd_cn-shanghai:v0.0.1-20221122114614]
[OK] Pushing image [registry-vpc.cn-shanghai.aliyuncs.com/eas/test_eascmd_cn-shanghai:v0.0.1-20221122114614]
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Service is running

The model service requires several minutes to deploy. You can go to the Elastic Algorithm Service (EAS) page to view the deployment status of the model service. For more information, see Model service deployment by using the PAI console.

After deployment, you can also use the EASCMD client to manage the model service. For more information, see Run commands to use the EASCMD client.

Use DSW

The DSW module of PAI is a cloud-based, interactive development environment that provides high-performance GPUs for deep learning. DSW integrates the EASCMD client. To deploy a model to EAS in DSW after model training, perform the following steps:

Enter DSW.
1. Log on to the PAI console.
2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to use.
3. In the upper-left corner of the page, select the region where you want to use PAI.
4. In the left-side navigation pane, choose Model Development and Training > Interactive Modeling (DSW).
5. (Optional) On the Interactive Modeling (DSW) page, enter the instance name or a keyword in the search box to search for a DSW instance.
6. Find the DSW instance and click Open in the Actions column.
Download the latest 64-bit Linux version of the EASCMD client and upload the downloaded file to a directory on the Notebook interface.
- For information about how to download the EASCMD client, see Download the EASCMD client and complete user authentication.
- For information about how to upload data files, see Upload or download data files.
In the top navigation bar of the DSW interface, click the Terminal tab to launch Terminal.
In Terminal, go to the directory where the EASCMD client is uploaded. Convert the EASCMD client into an executable file and use the AccessKey pair of your Alibaba Cloud account to complete identity authentication. For more information, see Download the EASCMD client and complete identity authentication.
Create a JSON file and upload it to the directory that contains the EASCMD client.
- For information about how to create a JSON file, see Run commands to use the EASCMD client.
- For information about how to configure the parameters in a JSON file, see Parameters of model services.
- For information about how to upload a JSON file, see Upload or download data files.
Run the following command in the directory that contains the JSON file to deploy the service:
```
./eascmd64 create <service_desc_json>
```
Replace <service_desc_json> with the name of the JSON file that you created.
The model service requires several minutes to deploy.

References

You can go to the Elastic Algorithm Service (EAS) page to view the deployment status of a model service and manage the model service. For more information, see Model service deployment by using the PAI consoles.
EAS provides multiple deployment methods. You can select a method based on your business requirements. For more information, see Overview.
After you deploy a service, you can use an automatic stress testing tool to test the service performance. For more information, see Automatic stress testing.
If the processors provided by EAS cannot meet your model deployment requirements, you can create a custom processor based on the processor development standards. For more information, see Deploy services by using custom processors.