If you specify only a single instance type when you deploy a service in Elastic Algorithm Service (EAS), resources of the specified type may be insufficient and the service cannot be created. To address this issue, EAS allows you to specify multiple instance types when you create a service. Then, the system prepares resources based on the instance types that you specify in the configuration file. This reduces the chances of insufficient resources in scenarios in which you specify only a single instance type. This topic describes how to specify multiple instance types.
Prerequisites
If you want to specify multiple instance types by using a client, make sure that the EASCMD client is installed, and identity authentication is complete. For more information, see Download the EASCMD client and complete identity authentication.
Limits
You can specify multiple instance types only for EAS services deployed in public resource groups.
Specify multiple instance types when you deploy a service
You can specify multiple instance types when you create a service. You can also update an existing service to configure multiple instance types.
The following example shows how to configure multiple instance types in the Platform for AI (PAI) console and on the EASCMD client.
Resource type | Machine model | Instance type | Bid price |
Preemptible | 8vcpu+16GB | ecs.c8i.2xlarge | 1 |
Common | 4vcpu+8GB | ecs.c8i.xlarge | None |
Configure multiple instance types in the PAI console
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).
Create a service: On the Inference Service tab, click Deploy Service. On the Deploy Service page, click Custom Deployment in the Custom Model Deployment section.
Update a service: On the Inference Service tab, find the desired service and click Update in the Actions column.
In the Resource Deployment section of the page that appears, configure the parameters described in the following table. For information about other parameters, see Deploy a model service in the PAI console.
Parameter
Description
Resource Type
Select Public Resources.
Deployment Resources
Resource Type: Select an instance type.
Bidding: If you turn on Bidding, preemptible instances are used. If you turn off Bidding, standard instances are used.
Bid Price: You need to specify a bidding price only for preemptible instances. The system automatically bids for preemptible instances based on the bidding price.
If the resources required to create the instance are sufficient and your bidding price is greater than or equal to the market price of the instance, the instance can be created.
Specify multiple instance types:
Click Add to add multiple instance types. You can add up to 10 instance types. Resources are prepared based on the sequence in which the instance types are configured.
Preemptible Instance Protection Period
The configuration takes effect only when you turn on Bidding. The following configuration methods are supported:
1-Hour Protection Period: You can specify a 1-hour protection period for a preemptible instance. During the protection period, the system ensures your access to the instance.
No Fixed Protection Period: The preemptible instance does not have a protection period. No protection period of a fixed time range is provided to ensure the use of resources.
For more information about preemptible instances, see Specify preemptible instances.
Click Deploy or Update. After the value of the Service Status parameter changes to Running, the service is deployed.
Configure multiple instance types on the EASCMD client
Create a service configuration file named
service.json. Sample content:{ "name": "service_name", "model_path": "http://examplebucket.aliyuncs.com/models%2Fmnist_saved_model.tar.gz", "processor": "tensorflow_cpu_1.12", "metadata": { "instance": 1 }, "cloud": { "computing": { "instances": [ { "type": "ecs.c8i.2xlarge", "spot_price_limit": 1 }, { "type": "ecs.c8i.xlarge" } ], "disable_spot_protection_period": false } } }The following table describes the parameters that are required in the service configuration file. For information about other parameters, see Run commands to use the EASCMD client.
Parameter
Description
instance
The number of service instances. In this example, one instance is specified in the JSON configuration file.
instances
One or more instance types that you can specify. If resources are insufficient for the first instance type, the system prepares resources for the next instance type.
type: the instance type.
spot_price_limit: your highest bidding price for preemptible instances. This parameter is optional.
If you configure this parameter, preemptible instances whose prices are less than the value specified by this parameter are used. Unit: USD. The pay-as-you-go billing method is supported.
If you leave this parameter empty, the instance type is a standard pay-as-you-go instance.
disable_spot_protection_period
Specifies whether to disable the instance protection period. Default value: false. Valid values:
false: The preemptible instance has a 1-hour protection period after the instance is created. During the protection period, even if the market price is higher than the bidding price, the instance cannot be released.
true: The preemptible instance does not have a protection period. Instances without a protection period are 10% cheaper than instances that have a protection period.
Run the following command in the directory in which the JSON file is located. For more information, see Run commands to use the EASCMD client.
In this example, a Windows 64 server is used.
Create a service
eascmdwin64.exe create <service.json><service.json>: the name of the JSON file that you created.
Update a service
eascmdwin64.exe modify <service_name> -s <service.json>Note:
<service_name>: the name of the EAS service that you want to manage.
<service.json>: the name of the JSON file that you created.