Platform For AI:Configure a custom monitoring and scaling metric

Limits

• You can use only the queries per second (QPS) and CPU Utilization metrics to configure auto scaling.

• You can specify the custom metric information by using the metrics field only when you deploy a service by using a custom image or custom processor.

Step 1: Configure an operation that is used to periodically report metrics

Before you deploy a service, you need to prepare a custom image or custom processor, and specify the custom metric and the operation that reports the metric in the configuration file. For more information, see Deploy a model service by using a custom image or Deploy services by using custom processors.

For example, if the POST request method in your code is http://localhost:8080/api/builtin/realtime_metrics, then the following code block provides a sample request body:

[
    {
        "name": "qps",
        "tags": {
            "status": "200"
        },
        "value": 20
    },
    {
        "name": "qps",
        "tags": {
            "status": "400"
        },
        "value": 13
    }
]

In the preceding example, a metric named qps is uploaded. The average QPS is 20 when the status code is 200, and the average QPS is 13 when the status code is 400. You can use the tags field in scenarios in which you need to classify metrics. Otherwise, you do not need to specify the tags field. Example:

[
    {
        "name": "qps",
        "value": 20
    }
]

Use your own code to define the operation that periodically reports the custom metric.

Step 2: Deploy a service

After you define a custom metric, you must specify information such as the metric name and type by using the metrics field in the JSON file of the service when you deploy the service. This way, the service can report the metric. EAS initializes and builds a dashboard on the monitoring page based on the metric information that you specify during service deployment. Perform the following steps:

1. Prepare a JSON file for the service configuration.

   Add the metrics field in the file to specify information such as the metric name and type. The following code provides a sample JSON file of using a custom image to deploy a service:

   {
     "name": "metrics_test",
     "containers": [
       {
         "image": "registry-vpc.cn-chengdu.aliyuncs.com/eas/eas-image-****:metrics",
         "command": "python3 -u /image.py",
         "port": 5000
       }
     ],
     "metrics": [
       {
         "name": "qps",
         "tags": "status"
       }
     ],
     "metadata": {
       "instance": 1,
       "cpu": 2,
       "memory":1000
     }
   }

   In the preceding code, set the image parameter to the custom image that you prepared. The following table describes the parameters of the metrics field. For more information about other parameters, see Parameters of model services.

   Parameter	Required	Description
   name	Yes	The name of the custom metric that you want to report in the service. The service monitoring page creates a monitoring dashboard based on the metric name. The metric name is also used to configure auto scaling.
   tags	No	The tags name of the metric. You can use tags to classify the metric. For example, you can classify the QPS based on the status code.

   The preceding configuration is an example when you use a Docker image to deploy a service. You can use a custom image to build your business logic. The configuration of the metrics field defines a metric named qps. The monitoring data is valid for a week. Expired data is automatically cleared. The configuration of the metrics field also defines a tag named status, which is used to report different information of the metric qps. By using the previous sample code, you can use the metric qps to collect data about the number of requests when the status code varies.

2. Use the configuration file to deploy the model service.

   1. Go to the Deploy Service page. For more information, see Model service deployment by using the PAI console.

   2. In the Configuration Editor section, click JSON Deployment and paste the content of the JSON file to the input box.

   3. Click Deploy.

Step 3: Display metric data

After you deploy the service, the system reports the metric data based on your configuration. EAS provides an on-premises API operation to receive the metric data.

• EAS also uses the received data for auto scaling.

• Due to the large amount of data, the data is collected and displayed within one minute after it is reported. The data that is displayed on the monitoring page is accurate to minute.

The following section provides an example of how to query the custom metric qps:

1. On the EAS-Online Model Services page, click the name of the service that you want to view to go to the Service Details page.

2. On the Service Monitoring tab, select the custom metric from the drop-down list in the upper-left corner. The dashboard of the custom metric appears.

   In the dashboard, you can view the monitoring data of the custom metric. The data includes the following parts:

   • The average metric data of all instances in the service.

   • The metric data of each instance in the service.

   

Step 4: Configure auto scaling

You can use the custom metric to configure service auto scaling. The configuration method is the same as using default metrics. For more information, see Enable or disable the horizontal auto-scaling feature.

The following section provides an example of using the custom metric qps to configure auto scaling.

• Use the on-premises client to enable auto scaling. For more information about the instructions, see the "Method 2: Manage the horizontal auto scaling feature by using a client" section in the Enable or disable the horizontal auto-scaling feature topic.

  • Sample code when no tags are configured for the custom metric:

    eascmd autoscale service_name -Dmin=1 -Dmax=10 -Dstrategies.custom[qps]=3

    The configuration indicates that the service scales out when the average value of QPS exceeds 3, and scales in when the average value of QPS is smaller than 3.

  • Sample code when a tag is configured for the custom metric:

    eascmd autoscale service_name -Dmin=1 -Dmax=10 -Dstrategies.custom[qps]@status[200]=3

    This configuration indicates that when the status code of QPS is 200, the service scales out when the average value of QPS exceeds 3, and scales in when the average value of QPS is smaller than 3.

• Use the console to enable auto scaling. For more information about the instructions, see the "Method 1: Manage the horizontal auto scaling feature in the console" section in the Enable or disable the horizontal auto-scaling feature topic.

  • If no tags is configured for the custom metric, in the Custom Scaling Metric section of the Auto Scaling Settings dialogue box, set the metric name to custom[qps] and the metric value to 3.

  • If a tag is configured for the custom metric, in the Custom Scaling Metric section of the Auto Scaling Settings dialogue box, set the metric name to custom[qps]@status[200] and the metric value to 3.