Introduction to offline prediction - Platform For AI - Alibaba Cloud Documentation Center

The prediction component can use models trained by EasyVision to perform offline prediction in Platform for AI (PAI). This topic describes the input data formats and PAI command parameters for offline prediction.

Overview

EasyVision can read data from and write results to MaxCompute tables. EasyVision can also read Object Storage Service files for prediction and then write the results to the OSS files. The offline prediction process can be viewed as an assembly line process. Each atomic operation can be asynchronously and concurrently processed in multiple threads on each worker node. During the I/O operation, each worker node obtains input data by using shards and write data to corresponding output shards. For example, when images are read from a table for model prediction, the system splits data in the input table based on the number of worker nodes. Each worker node reads its own data, decodes Base64 data, performs model prediction, and writes the results to the output table. Base64 decoding and model prediction are asynchronously performed by multiple threads. This can fully utilize the CPU and GPU computing power for concurrent processing. The following figure shows the Base64 decoding and model prediction processes.

EasyVision provides video-level prediction models to process video data. You can also call image-related models to predict video frames. The offline processing framework of EasyVision automatically decodes video data, predicts single-frame images, and summarizes all video frame results.

The default prediction code provided by EasyVision cannot be used to load trained models for offline prediction EasyVision allows you to specify custom prediction code and reuse the existing I/O feature of ev_predict. You can perform offline prediction for your model by downloading and decoding data. EasyVision also allows you to insert a custom process before prediction. Input data is processed before being sent to the predictor. For more information, see Custom input data.

Input data formats

Read data from MaxCompute tables

The input table can contain one or more columns. One column is for the Base64-encoded URLs or binary data of images. The data in this column is of the string type. Sample schema:

+------------------------------------------------------------------------------------+
| Field           | Type       | Label | Comment                                     |
+------------------------------------------------------------------------------------+
| id              | string     |       |                                             |
| url             | string     |       |                                             |
+------------------------------------------------------------------------------------+

Read data from OSS files
Each column of the OSS input file is a URL or an OSS endpoint. Example:
```
oss://your/path/to/image.jpg
http://your.path/to/image.jpg
```
Custom input data
When data is read from a MaxCompute table, only the URL or Base64-encoded data of the image is obtained. When data is read from an OSS file, only the URL of the image is obtained to facilitate the download and decoding of the image. The preceding methods can obtain only NumPy arrays of the image. The {"image": np.ndarray} format is used by various processes and predictors during prediction. As the number of users who use custom predictors and processes increases, the single input format cannot meet business requirements. Therefore, the OSS reading mode is modified to support custom data formats.
The custom format can be the original OSS file format or JSON string format. Each line in the input files is a JSON string. You can enter multiple key-value pairs. All key-value pairs are saved in a dictionary and transferred to custom predictors and processes. You can easily obtain the corresponding values based on custom keys.
If the value is an OSS path or URL, the system automatically uses multiple threads to download the file content and converts the value into a Python-formatted object. You can call the file method such as read() or readlines() to obtain the file content. If the value points to a file with an image extension, the system automatically decodes the image. You can obtain the value from the input_data dictionary based on keys. The value is of the numpy.ndarray type.
Sample input data:
```
{"image":"http://your/path/to/image.jpg", "prior":"oss://your/path/to/prior.txt", "config": {"key1":1, "key2":"value2"}}
{"image":"http://your/path/to/image.jpg", "prior":"oss://your/path/to/prior.txt", "config": {"key2":1, "key2":"value2"}}
```
The preceding input data is converted into data in the input_data dictionary containing the following fields:
- image: the decoded data of an image.
- prior: the file-formatted object.
- config: a dictionary of JSON strings.
The input_data dictionary is in the following format. For all custom process and predictor classes, you can query image data by using keys.
```
input_dict = {
  "image": np.ndarray,
  "prior" : file_like_object,
  "config": {"key1":1, "key2":"value2"}
}
```
Important
All built-in predictors of EasyVision use the image key to obtain input images. If you want to use a custom input format to call the built-in predictors of EasyVision, you must use the image key for image data.

Parameters

Parameter	Required	Description	Type	Default value
model_path	Yes	The OSS path of the model. Example: "oss://your_bucket/your_model_dir".	STRING	N/A
model_type	Yes	The type of the model. Valid values: feature_extractor: feature extraction classifier: image classification multilabel_classifier: multi-label classification detector: object detection text_detector: text detection text_recognizer: text line recognition text_detection_recognition: text detection and recognition text_spotter: end-to-end text recognition segmentor: semantic image segmentation self_define: custom prediction If you set the model_type parameter to self_define, the predictor class that is specified by the user_predictor_cls parameter is loaded.	STRING	N/A
buckets	Yes	The information about the OSS buckets. If you use a custom model, you must specify the OSS bucket information to store the model. Example: "oss://{bucket_name}.{oss_host}/{path}".	STRING	N/A
arn	Yes	The Alibaba Cloud Resource Name (ARN) of the RAM role that has the permissions to access OSS resources. For more information about how to obtain the ARN, see the "I/O parameters" section of the Parameters of PAI-TensorFlow tasks topic.	STRING	N/A
feature_name	No	The name of the feature that you want to extract. This parameter is required only if you set the model_type parameter to feature_extractor. Example: resnet_v1_50/logits.	STRING	Empty string ("")
input_table	No	The name of the input table. For example, you can use a non-partitioned table named "odps://prj_name/tables/table_name" or a partitioned table named "odps://prj_name/tables/table_name/pt=xxx".	STRING	Empty string ("")
image_col	No	The name of the column that contains the image data.	STRING	"image"
image_type	No	The format of the image data. Valid values: base64: indicates that the table stores the image data in the Base64 format. url: indicates that the table stores the URLs or OSS paths of the images.	STRING	"base64"
reserved_columns	No	The names of reserved data columns. Separate multiple names with commas (,). Example: "col1,col2,col3".	STRING	Empty string ("")
result_column	No	The name of the result column.	STRING	"prediction_result"
output_table	No	The output table. The value format is the same as the value format of the input_table parameter. If the table does not exist, the system automatically creates an output table and creates partitions for the table. You can also create a partitioned table as the output table before you perform the prediction.	STRING	Empty string ("")
lifecycle	No	The lifecycle of the output table.	INT	10
num_worker	No	The number of prediction worker nodes. You can increase the number of worker nodes to accelerate offline prediction.	INT	2
cpuRequired	No	The number of CPUs for a worker node. A value of 100 indicates one CPU.	INT	1600
gpuRequired	No	The number of GPUs for each worker node. A value of 100 indicates one GPU. You can use up to 100 GPUs for a worker node. A value of 0 indicates that a CPU cluster is used.	INT	100
input_oss_file	No	The path of the input OSS file. Each line in the input file can be in one of the following formats: The OSS path or URL of an image that you want to predict. Example: oss://your_bucket/filelist.txt. A JSON string. For more information, see Custom input data.	STRING	Empty string ("")
output_oss_file	No	The path of the output OSS file that is used to store the prediction results. The system may generate multiple result files and merge the result files into an output OSS file. The result files are prefixed with the name of the output OSS file. The number of result files is the same as the number of worker nodes, which is specified by the num_worker parameter.	STRING	Empty string ("")
output_dir	No	The folder in which the output files are stored. Example: "oss://your_bucket/dir". If you use a custom output format, all result image files are saved to this folder.	STRING	Empty string ("")
user_resource	No	The path of the uploaded resource, which can be a TAR.GZ, ZIP, or Python file. OSS paths and HTTP URLs are supported. Examples: oss://xxx/a.tar.gz and http://example.com/c.zip.	STRING	Empty string ("")
user_predictor_cls	No	The module path of the custom predictor class. For example, if you implement Predictor A in module.py, the module path of Predictor A is module.A.	STRING	Empty string ("")
user_process_config	No	The configurations of the custom process class. You can use the following fields to configure the process class. You can also configure other custom fields. job_name: the name of the custom process class. num_threads: the number of concurrent threads that are used by the custom process class. batch_size: the batch size of the data that you want to process. user_process_cls: the module path of the custom process class. For example, if you implement Process A in module.py, the module path of Process A is module.A. Example: '{["job_name":"myprocess","user_process_cls": module.ClassA"num_threads":2, "batch_size":1]}'	JSON	Empty string ("")
queue_size	No	The length of the cache queue.	INT	1024
batch_size	No	The batch size for prediction.	INT	1
preprocess_thread_num	No	The number of concurrent threads that are used for preprocessing, such as image decoding and download.	INT	4
predict_thread_num	No	The number of concurrent threads that are used for prediction.	INT	2
is_input_video	No	Specifies whether the input is a video. Valid values: true false	BOOL	false
use_image_predictor	No	Specifies whether the predictor supports only image input.	BOOL	true
decode_type	No	The method that is used to decode the videos. Valid values: 1: intra only 2: keyframe only 3: without bidir 4: decode all	INT	4
sample_fps	No	The number of frames that are extracted for sampling per second.	FLOAT	5
reshape_size	No	The size of the output frames in pixels. If you set this parameter to -1, the frames are not resized.	INT	-1
decode_batch_size	No	The number of frames that are decoded at the same time.	INT	10
decode_keep_size	No	The number of overlapped frames in different batches.	INT	0
enableDynamicCluster	No	Specifies whether to enable the dynamic cluster feature. If the feature is enabled, the failover of a single worker node is allowed. If task exceptions frequently occur, we recommend that you enable the dynamic cluster feature. Valid values: true false	BOOL	false
useSparseClusterSchema	No	If you set the enableDynamicCluster parameter to true, you must also set the useSparseClusterSchema parameter to true. Valid values: true false	BOOL	false