Platform For AI:Appendix: Status codes

Causes:

• The request format is invalid. This may occur with general processors such as TensorFlow and PyTorch.

• The request encounters an exception. This may occur with custom processors.

  Note

  You can also customize status codes for custom processors.

Solutions

1. Use the token that was automatically generated or manually specified when the service was deployed. When a service is deployed, the system automatically generates a token that you can use to access the service. To manually specify a token the first time you deploy a service, add the Token parameter to the service configuration files to specify a fixed token value. This way, the token value does not change even if you update the service.

2. Obtain the token value in the console. Log on to the PAI console, find the workspace that you want to manage, and go to the details page of the workspace. In the left-side navigation pane, click Model Serving (EAS). On the Elastic Algorithm Service page, find the service that you want to manage, click Debug in the Operating column, and then obtain the token value of the service. You can also run an eascmd desc service_name command to obtain the token value.

3. Access the service by using the obtained token value.

   • Add the token value to the request by using an HTTP header when you run a curl command to test the service. Sample command: curl -H 'Authorization: NWMyN2UzNjBiZmI2YT***' http:// xxx.cn-shanghai.aliyuncs.com/api/predict/echo

   • Invoke a SetToken() function when you use an SDK to access the service. For more information, see SDK for Java.

Causes: The requests failed to be processed within the specified timeout duration. All requests are given the same timeout duration. If a request is not processed within the specified time, the request times out, the TCP connection is closed, and status code 408 is returned. By default, the timeout period is set to 5 seconds. This limit can be modified by using the metadata.rpc.keepalive parameter in the JSON file used in the create command. For more information about the create command, see Create a service.

Causes: The header fields of the request are too large. When an instance receives a request, it puts the request in a queue. The request is processed when instance workers become available. By default, each instance can have a maximum of five workers. This limit can be modified by using the metadata.rpc.worker_threads parameter in the JSON file used in the create command. For more information about the create command, see Create a service. If workers are occupied for extended periods of time, requests in the queue begin to pile up. When the queue reaches its upper limit, new requests are denied and status code 450 is returned. This ensures service availability by keeping the requests to a manageable number and avoiding runaway response times. By default, each queue can hold a maximum number of 64 requests. This limit can be modified by using the metadata.rpc.max_queue_size parameter in the JSON file used in the create command. For more information about the create command, see Create a service.

Solutions

• Reschedule the request to an idle instance to prevent service interruption. This solution is applicable to scenarios where status code 450 is returned for a small number of requests. If you use this method on a large number of requests, the throttling measure fails to serve its purpose and may lead to cascading failures.

• Debug code for the processor. If an exception occurs on the processor due to a code error, all workers of an instance become unavailable due to their deadlock states and status code 450 is returned for all requests.