You can use multimedia analysis SDK for Python provided by Platform for AI (PAI) to call different algorithm-powered services. This topic describes how to use multimedia analysis SDK for Python to call different model services. This topic also provides sample requests and responses.
Step 1: Install multimedia analysis SDK for Python
The environment relies on Python 3 or later. Run the following command to install multimedia analysis SDK for Python:
wget https://ai-service-data.oss-cn-beijing.aliyuncs.com/python-sdk/ai_service_python_sdk-1.1.4-py3-none-any.whl
pip install ai_service_python_sdk-1.1.4-py3-none-any.whl
Step 2: Initialize the client
Run the following command to initialize the environment:
from ai_service_python_sdk.client.api_client import ApiClient
client = ApiClient('<HOST>', '<YOUR-APPID>', '<YOUR-TOKEN>')
Modify the following parameters based on your business requirements.
Parameter | Description |
<HOST> | The server address is |
<YOUR-APPID> | After you activate multimedia analysis, you can view AppID on the multimedia analysis page. For more information, see Multimedia Analysis. |
<YOUR-TOKEN> | After you activate multimedia analysis, you can view Token on the multimedia analysis page. For more information, see Multimedia Analysis. |
Examples
Image quality evaluation
You can evaluate the quality of an image. The model service returns a floating-point number from 0 to 100. Sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = ''
# The response settings of the model. If the model does not have response parameters, you can specify an empty dictionary.
configure = {}
# The model service that you want to call.
response = ai_service_api.image_qa(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the parameters in the preceding statement.
Parameter | Type | Required | Description |
image_url | STRING | Yes | The URL of the image. We recommend that the size of the image is no larger than 5 MB and the resolution of the image is equal to or higher than 240 × 240. |
model_name | STRING | Yes | By default, an empty string is specified to call a general-purpose model. If the general-purpose model cannot meet your business requirements, contact your account manager to obtain a custom model. You can specify the name of a custom model to call the custom model. |
configure | Dict | Yes | The response settings of the model. Specify an empty dictionary for the model service. |
Sample output:
{
"code":"OK",
"data":{
"image_height":214,
"image_width":398,
"iqa_result":66.88
},
"message":"success",
"request_id":"1d4572a1-2800-4213-9e9c-33f9fa7e****"
}
The following table describes the response parameters.
Parameter | Description |
request_id | The request ID, which is of the STRING type. |
code | The request status code, which is of the STRING type. |
message | The detailed status information about the request, which is of the STRING type. Sample output:
For more information, see Error codes. |
data | The content returned by the model service, which is of the DICT type. Parameter descriptions:
|
Face attribute analysis
You can obtain the generated face-related attributes, such as the face shape, hair color, hairstyle, and appearance. The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = ''
# The response settings of the model. If the model does not have response parameters, you can specify an empty dictionary.
configure = {}
# The model service that you want to call.
response = ai_service_api.face_attr_image(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the parameters in the preceding statement.
Parameter | Type | Required | Description |
image_url | STRING | Yes | The URL of the image. The face section in the image must have a resolution that is higher than 100 × 100. |
model_name | STRING | Yes | By default, an empty string is specified to call a general-purpose model. If the general-purpose model cannot meet your business requirements, contact your account manager to obtain a custom model. You can specify the name of a custom model to call the custom model. |
configure | Dict | Yes | The response settings of the model. Specify an empty dictionary for the model service. |
Sample output. Take note that some parameters contain untranslated Chinese text due to technical reasons.
{'code': 'OK',
'data': {'attr': [{'FaceShape': {'三角脸': 0.0013,
'圆脸': 0.0147,
'心形脸': 0.0405,
'方形脸': 0.0001,
'椭圆脸': 0.9266,
'菱形脸': 0.0027,
'长脸': 0.0141},
'FemaleHairStyle': {'BangsType': {'中分刘海': 0.5932,
'刘海编发': 0.0058,
'斜刘海': 0.0575,
'无刘海': 0.3078,
'空气刘海': 0.0143,
'齐刘海': 0.0213},
'FemaleCurlyHairType': {'云朵卷': 0.1289,
'大波浪': 0.2526,
'小波浪': 0.084,
'氧气卷': 0.2027,
'空气卷': 0.1434,
'羊毛卷': 0.1539,
'蛋卷': 0.0345},
'FemaleHairType': {'卷发': 0.1941,
'盘发': 0.0112,
'直发': 0.7947},
'FemaleLongShortHair': {'中长发': 0.2807,
'短发': 0.0346,
'长发': 0.6846},
'扎发': 0.3054,
'编发': 0.3453},
'HairColor': {'其他颜色': 0.0001,
'咖啡色': 0,
'奶奶灰': 0,
'栗色': 0.0025,
'棕色': 0.0622,
'渐变色': 0.0004,
'酒红色': 0.0001,
'金色': 0,
'黄色': 0,
'黑色': 0.9347},
'MaleHairStyle': {'分发': 0.9939,
'圆寸': 0,
'子弹头': 0.0033,
'平头': 0,
'板寸': 0.001,
'毛寸': 0.0008,
'碎发': 0,
'背头': 0.0001,
'莫西干': 0.0006},
'bbox': [91, 0, 430, 452],
'face_score': 3.2431,
'index': 0}]},
'message': 'success',
'request_id': '9b2f2c0e-d2c0-45f8-b922-058d00edc9ad'}
The following table describes the response parameters.
Parameter | Description |
request_id | The request ID, which is of the STRING type. |
code | The request status code, which is of the STRING type. |
message | The detailed status information about the request, which is of the STRING type. Sample output:
For more information, see Error codes. |
Data | The content returned by the model service, which is of the DICT type. For more information about the response parameters, see Data parameters. |
Table 1. Data parameters
Parameter | Description |
attr | The attribute content of the image, which is of the LIST type. Each element in the list indicates one face attribute. Note If the list is empty, no faces are detected. |
FaceShape | The shape of the face, which is of the DICT type. |
FemaleHairStyle | The female hairstyle type, which is of the DICT type. |
BangsType | The female bangs type, which is of the DICT type. |
FemaleCurlyHairType | The female curl type, which is of the DICT type. If the FemaleHairType parameter is most likely to be set to a curly hairstyle, the value of this parameter is read. |
FemaleHairType | The female hair type, which is of the DICT type. |
FemaleLongShortHair | The female hair length, which is of the DICT type. |
Tied hair | Specifies whether the hair of the female is tied. The value is of the FLOAT type. If the value of this parameter is greater than or equal to 0.5, the hair is tied. |
Braided hair | Specifies whether the hair of the female is braided. The value is of the FLOAT type. If the value of this parameter is greater than or equal to 0.5, the hair is braided. |
HairColor | The hair color, which is of the DICT type. |
MaleHairStyle | The male hairstyle, which is of the DICT type. |
bbox | The rectangle section in which the face resides, which is of the LIST type. The value is in the |
index | The ID of the face in the image. The value is of the INT type and starts from 0. |
Age analysis
You can identify the age interval of the major face in the image.
The model is a custom model. If you want to use the model, contact your account manager and provide the AppId for configuration.
Sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = 'fairface'
# The response settings of the model. If the mode does not have response parameters, you can specify an empty dictionary.
configure = {}
# The model service to be called.
response = ai_service_api.face_attr_image(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the key parameters.
Parameter | Type | Required | Description |
image_url | STRING | Yes | The URL of the image. The face section in the image must have a resolution that is higher than |
model_name | STRING | Yes | The name of the custom model. |
configure | Dict | Yes | The response settings of the model. Specify an empty dictionary for the model service. |
Sample response:
{
"request_id":"10000",
"code":"OK",
"message":"success",
"data":{
"age_range":"3-9",
"age_raneg_prob":0.9525
}
}
Parameter | Description |
request_id | The request ID, which is of the STRING type. |
code | The request status code, which is of the STRING type. |
message | The detailed status information about the request, which is of the STRING type. Sample output:
|
data | The content returned by the model service, which is of the DICT type. Response parameters.
|
Multiple image labels
You can add multiple labels to an image. The model service can return top K labels with the highest probabilities and the probability of each label. The model service can also return high-dimensional features of the image. The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = ''
# The response settings of the model.
configure = {'tag_top_k': 5, 'output_embedding': False}
# The model service that you want to call.
response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the parameters in the preceding statement.
Parameter | Type | Required | Description |
image_url | STRING | Yes | The URL of the image. We recommend that the size of the image is no larger than 5 MB and the resolution of the image is equal to or higher than 240 × 240. |
model_name | STRING | Yes | By default, an empty string is specified to call a general-purpose model. If the general-purpose model cannot meet your requirements, you can contact the technical support of PAI to obtain a custom model. You can specify the name of a custom model to call the custom model. |
configure | Dict | Yes | The response settings of the model. The default value is Note If configure is set to None, the parameters in the configure list use the default settings. |
Configure parameters
Parameter | Type | Required | Description |
tag_top_k | INT | No | The top K labels. Valid values of K: 1 to 10. Default value: 10. |
output_embedding | BOOL | No | Specify whether to return the image embedding. Valid values:
|
Sample output. Take note that some parameters contain untranslated Chinese text due to technical reasons.
{'code': 'OK',
'data': {'image_height': 510,
'image_width': 516,
'tag_result': [{'score': 0.505, 'tag': '笑容'},
{'score': 0.493, 'tag': '女士'},
{'score': 0.475, 'tag': '美女'},
{'score': 0.438, 'tag': '微笑'},
{'score': 0.432, 'tag': '蜜茶棕'}]},
'message': 'success',
'request_id': 'c9e5aa33-4d8d-49ce-8991-bcbb5f12e43b'}
The following table describes the response parameters.
Parameter | Description |
request_id | The request ID, which is of the STRING type. |
code | The request status code, which is of the STRING type. |
message | The detailed status information about the request, which is of the STRING type. Sample output:
For more information, see Error codes. |
data | The content returned by the model service, which is of the DICT type. For more information about the response parameters, see Data parameters. |
Table 3. Data parameters
Parameter | Description |
tag_result | The labeling result, which is of the LIST type. The labels are sorted in descending order by score (probability). The length of the list varies based on the value of tag_top_k. |
score | The probability of the label, which is of the FLOAT type. The value is accurate to three decimal places. |
tag | The name of the label, which is of the STRING type. |
embedding_result | The 512-dimensional features of the image, which are of the STRING type. The features are separated by commas (,). This parameter is returned only when output_embedding is set to True. |
image_height | The height of the image in pixels, which is of the INT type. |
image_width | The width of the image in pixels, which is of the INT type. |
Labeling AI paintings
You can add multiple labels to various types of images that are generated by Stable Diffusion models to improve the training performance of image generation. The output is a synthesized caption and a score related to the original set of labels.
This model is a custom model. If you want to use the model, provide the AppId to your account manager for technical support.
The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = 'tagger'
# The response settings of the model.
configure = {'threshold': 0.5, 'tagger_type': "SmilingWolf/wd-v1-4-convnextv2-tagger-v2"}
# The model service to be called.
response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the key parameters.
Parameter | Description | Type | Required |
image_url | The URL of the image. We recommend that the size of the image is no larger than 5 MB and the resolution of the image is equal to or higher than 240 × 240. | STRING | Yes |
model_name | The name of the model. Set the parameter to | STRING | Yes |
configure | The response settings of the model. The default is | Dict | Yes |
Sample response:
{
"data": {
"caption": "sensitive, 1girl, solo, long hair, looking at viewer, smile, black hair, brown eyes, scarf, lips, realistic",
"probs": [1.1920928955078125e-07, ............, 3.5762786865234375e-07, 4.76837158203125e-07],
}
"request_id": "12***",
"code": "OK",
"message": "success"
}
Parameter | Description | Type |
request_id | The ID of the request. | STRING |
code | The request status code. | STRING |
message | The detailed status information about the request. Examples:
| STRING |
data | The content returned by the model service. For more information, see Data parameters. | Dict |
Portrait figure modification
You can modify the figure in the image. You can upload a portrait image and modify the figure to be larger or slimmer by adjusting the degree parameter. For example, you can set the degree
parameter to a value that is greater than 0 to slim the figure.
This model is a custom model. If you want to use the model, provide the AppId to your account manager for technical support.
Sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the model that you want to use.
model_name = 'body_reshaping'
# If you want to enlarge the figure, we recommend that you set the degree parameter to a value between -2.5 and -1.5.
configure = {'degree': -1.5}
# The model service to be called.
response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the request parameters.
Parameter | Description | Type | Required |
image_url | The URL of the image. We recommend that the size of the image is no larger than 5 MB and the resolution of the image is equal to or higher than 240 × 240. | STRING | Yes |
model_name | The name of the model. Set the value to | STRING | Yes |
configure | The response settings of the model. The default is | Dict | Yes |
Sample response:
{
"code":"OK",
"data":"imByev+I3vrryTGN****",
"message":"success",
"request_id":"1d4572a1-2800-4213-9e9c-33********"
}
The following table describes the response parameters.
Parameter | Description | Type |
request_id | The ID of the request. | STRING |
code | The request status code. | STRING |
message | The detailed status information about the request. Examples:
| STRING |
data | The Base64-encoded image that is returned by the model service. | STRING |
Watermark removal
You can remove the watermarks of an image.
This model is a custom model. If you want to use the model, provide the AppId to your account manager for technical support.
Sample request:
from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi
# The URL of the image.
image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
# The name of the custom model.
model_name = 'watermark_remover'
# Specify an empty dictionary.
configure = {}
# The model service to be called.
response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The return value.
data = response.data
# Display the return value.
print(response)
The following table describes the parameters.
Parameter | Description | Type | Required |
image_url | The URL of the image. We recommend that the size of the image is no larger than 5 MB and the resolution of the image is equal to or higher than 240 × 240. | STRING | Yes |
model_name | The name of the model. Set the value to | STRING | Yes |
configure | The response settings of the model. The default is | Dict | Yes |
Sample response:
{
"code":"OK",
"data":"imByev+I3vr****",
"message":"success",
"request_id":"1d4572a****"
}
The following table describes the returned parameters.
Parameter | Description | Type |
request_id | The ID of the request. | STRING |
code | The request status code. | STRING |
message | The detailed status information about the request. Examples:
| STRING |
data | The Base64-encoded image that is returned by the model service. | STRING |
Post categorization and labeling
You can label posts that include multi-modal content. For example, you can categorize and label content that consists of texts and images or texts and videos to increase the labeling accuracy. The model service also returns the embedding results of high-dimensional features.
The model is a custom model. If you want to use the model, contact your account manager and provide the AppId for configuration.
Sample request:
from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi
ai_service_api = AiServiceVideoApi(client)
# The name of the model that you want to use.
model_name = 'multi_modal_tag'
# The video URL. If no video URL is available, specify an empty string.
video_url = 'https://YOUR_VIDEO.mp4'
# The image URLs. If no image URL is available, specify an empty array.
images = [
'https://YOUR_IMAGE_0.jpg',
'https://YOUR_IMAGE_1.jpg',
'https://YOUR_IMAGE_2.jpg'
]
# Text content.
text ='Text content'
# The response settings of the model.
configure = {
'output_classification': True,
'classification_top_k': 1,
'output_tag': True,
'tag_top_k': 5,
'output_embedding': True
}
# The model service to be called.
response = ai_service_api.multimodal_classify(video_url, images, text, model_name, configure)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The detailed status information about the request.
message = response.message
# The content returned by the request.
data = response.data
# job id
job_id = response.data['job_id']
# Display the return value.
print(response)
The following table describes the key parameters.
Parameter | Type | Required | Description |
video_url | STRING | Yes | The video URL. If no video URL is available, specify an empty string. |
images | List[STRING] | Yes | The image URLs. If no image URL is available, specify an empty array. Separate multiple image URLs with commas (,). |
text | STRING | Yes | The text content, which consists of the title and the introduction. |
model_name | STRING | Yes | The name of the model that you want to use. |
configure | Dict | Yes | The additional configuration of the model, which is of the DICT type. For more information, see Configure parameters. |
Sample response:
{'code': 'OK',
'data': {'job_id': 1134***},
'message': 'success',
'request_id': '49b0c9c2-06be-4f30-9b9f-f04baecf4577'}
Obtain the job_id from the preceding returned results and use the following code to query the results.
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi
ai_service_job_api = AiServiceJobApi(client)
# Query results.
result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>)
# Display the return value.
print(result)
Replace <YOUR-JOB-ID> with the job_id that you obtained.
Sample response:
The following result is returned if the job is running:
{'code': 'OK', 'data': {'job': {'Result': '', 'app_id': 'YOUR_APPID', 'create_time': '2023-10-18T14:32:42.619+08:00', 'id': 1134206, 'message': 'video downloading', 'state': 1}}, 'message': 'success', 'request_id': '49b0c9c2-06be-4f30-9b9f-f04baecf4577'}
The following result is returned if the job is completed. Take note that some parameters contain untranslated Chinese text due to technical reasons.
{ 'code': 'OK', 'data': {'job': {'Result': '{"classification_result":[{"class":"旅游","score":0.67}], "duration":15.16, "embedding_result":"0.915,0.882,0.943,0.978,1.027,1.181,1.066,1.029,0.866,0.716,0.628,1.203,0.689,0.533,0.734,1.038,0.98,0.613,0.96,0.88,0.586,0.702,1.515,0.697,0.987,0.699,1.179,4.274,0.757,0.89,0.805,0.901", "tag_result":[{"score":0.56,"tag":"有字幕"},{"score":0.456,"tag":"航拍"},{"score":0.453,"tag":"西藏"},{"score":0.437,"tag":"交通工具"},{"score":0.429,"tag":"机械"}], "video_height":918, "video_width":552}', 'app_id': '202211140827340****', 'create_time': '2023-10-18T14:32:42.619+08:00', 'id': 113****, 'message': 'success', 'model_id': 0, 'request_id': '', 'state': 2, 'type': 'Video' } }, 'message': 'success', 'request_id': '139a40ad-fdf6-45de-b760-161c52ff3f3c' }
The following table describes the response parameters.
Parameter | Type | Description |
request_id | STRING | The ID of the request. |
code | STRING | The request status code. |
message | STRING | The detailed status information about the request. Possible values:
|
data | Dict | The content returned by the model service. For more information, see Data parameters. |
Video quality evaluation
You can evaluate the quality of a video. The model service returns a floating-point number from 0 to 100. You can use one of the following methods to call the model service:
Evaluate the quality of a single video. The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi video_url = 'https://your_video_url.mp4' ai_service_api = AiServiceVideoApi(client) # The name of the custom model. model_name = '' # The title of the video. video_title = '' # The response settings of the model. configure = {} # The model service that you want to call. response = ai_service_api.video_qa(video_url, model_name, video_title, configure) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The detailed status information about the request. message = response.message # The content returned by the model service. data = response.data # The ID of the job. job_id = response.data['job_id'] # Display the return value. print(response)
The following table describes the parameters in the preceding statement.
Parameter
Type
Required
Description
video_url
STRING
Yes
The URL of the video. We recommend that the size of the video is no larger than 100 MB and the resolution of the video is equal to or higher than 320 × 240.
video_title
STRING
Yes
The title of the video. Specify an empty string because the model service does not require you to specify a video title.
model_name
STRING
Yes
By default, an empty string is specified to call a general-purpose model.
If the general-purpose model cannot meet your business requirements, contact your account manager to obtain a custom model. You can specify the name of a custom model to call the custom model.
configure
Dict
Yes
The response settings of the model. Specify an empty dictionary for the model service.
Sample output:
{'code': 'OK', 'data': {'job_id': 10**}, 'message': 'success', 'request_id': '228235a6-db4f-466a-b20b-a4453f81****'}
Obtain job_id from the preceding returned response and specify the job ID in the following code to query the result.
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi ai_service_job_api = AiServiceJobApi(client) # Query the result. result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>) # Display the return value. print(result)
Replace <YOUR-JOB-ID> with the value of job_id returned by the model service, which is of the INT type.
The following result is returned if the job is running:
{'code': 'OK', 'data': {'job': {'Result': '', 'app_id': '202209070402410****', 'create_time': '2022-09-07T14:03:33.8+08:00', 'id': 1012, 'message': 'video downloading', 'state': 1}}, 'message': 'success', 'request_id': '5259ec57-e55c-4772-b01e-3deafd74****'}
The following result is returned if the job is completed:
{'code': 'OK', 'data': {'job': {'Result': '{"avg_fps":30,"duration":14,"height":1920,"single_img":0.74072265625,"video_score":20.57750291278764,"width":1080}', 'app_id': '202209290847310****', 'create_time': '2022-10-12T10:51:30.015+08:00', 'id': 1012, 'message': 'success', 'state': 2}}, 'message': 'success', 'request_id': '0b4f658f-cd96-4606-8d81-d2fe0a6d****'}
The following table describes the response parameters.
Parameter
Type
Description
request_id
STRING
The request ID, which is of the STRING type.
code
STRING
The request status code, which is of the STRING type.
message
STRING
The detailed status information about the request, which is of the STRING type. Sample output:
success
Invalid Input - video data error - load video failed
Invalid Input - json format error - json parse runtime exception
For more information, see Error codes.
data
Dict
The content returned by the model service, which is of the DICT type. For more information about the response parameters, see Data parameters.
Table 4. Data parameters
Parameter
Description
id
The ID of the job (job_id), which is of the INT type.
app_id
The AppId, which is of the INT type.
state
The status code of the job, which is of the INT type. Valid values:
0: The job is initializing.
1: The job is running.
2: The job is completed.
3: The job failed.
message
The execution information of the job, which is of the STRING type. Valid values:
success
video downloading
create_time
The time when the job was created, which is of the STRING type.
Result
The result returned by the model service, which is of the STRING type. For more information, see Result parameters.
Table 5. Result parameters
Parameter
Description
video_score
The quality score of the video, which is of the FLOAT type. Valid values: 0 to 100.
avg_fps
The average frame rate of the video, which is of the INT type.
duration
The duration of the video, which is of the INT type. Unit: seconds.
single_img
The difference between the images in the video, which is of the FLOAT type. Valid values: 0 to 1. A smaller value indicates a smaller difference. This means that the video is generated based on static images.
height
The height of the video in pixels, which is of the INT type.
width
The width of the video in pixels, which is of the INT type.
Evaluate the quality of multiple videos in a batch. The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi from ai_service_python_sdk.client.models.batch_video_request_videos import BatchVideoRequestVideos ai_service_api = AiServiceVideoApi(client) # Create a batch processing request (video URL + video title). videos = [ BatchVideoRequestVideos('https://your_video_1_url.mp4', ''), BatchVideoRequestVideos('https://your_video_2_url.mp4', '') ] # The name of the custom model. model_name = '' # The response settings of the model. configure = {} # The model service that you want to call. response = ai_service_api.batch_video_qa(videos, model_name, configure) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The detailed status information about the request. message = response.message # The content returned by the model service. data = response.data # The ID of the batch job. batch_id = response.data['batch_id'] # Display the return value. print(response)
Sample output:
{'code': 'OK', 'data': {'batch_id': 14}, 'message': 'success', 'request_id': '6d5ef1b5-c01f-4e63-9eec-150ddde9****'}
Obtain batch_id from the preceding returned response and use the batch ID in the following code to query the result:
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi ai_service_job_api = AiServiceJobApi(client) # Query the result. result = ai_service_job_api.get_batch_job_with_id(<YOUR-BATCH-ID>) # Display the return value. print(result)
Replace <YOUR-BATCH-ID> with the value of batch_id returned by the model service, which is of the INT type.
The following result is returned:
{'code': 'OK', 'data': {'jobs': [{'Result': '{"avg_fps":30,"duration":14.633333333333333,"height":1280,"single_img":0.486328125,"video_score":55.3773279862801,"width":720}', 'app_id': '202209290847310****', 'batch_id': 14, 'create_time': '2022-10-26T19:44:47.501+08:00', 'id': 44**, 'message': 'success', 'object': 'https://your_video_1_url.mp4', 'state': 2}, {'Result': '{"avg_fps":30,"duration":18.2,"height":568,"single_img":0.5791015625,"video_score":67.21071975649835,"width":320}', 'app_id': '202209290847310****', 'batch_id': 14, 'create_time': '2022-10-26T19:44:47.501+08:00', 'id': 4484, 'message': 'success', 'object': 'https://your_video_2_url.mp4', 'state': 2}]}, 'message': 'success', 'request_id': 'e99697ff-7d42-4c0a-a0a0-c6a0c0ba****'}
The data['jobs'] parameter returned by the batch job is an array. The following table describes the parameters in the array.
Parameter
Description
id
The ID of the data entry in the batch job, which is of the INT type.
batch_id
The ID of the batch job, which is of the INT type.
app_id
The AppId, which is of the INT type.
state
The status code of the job, which is of the INT type. Valid values:
0: The job is initializing.
1: The job is running.
2: The job is completed.
3: The job failed.
message
The execution information of the job, which is of the STRING type. Valid values:
success
video downloading
create_time
The time when the job was created, which is of the STRING type.
object
The URL of the video, which is of the STRING type.
Result
The result returned by the model service, which is of the STRING type.
NoteThe Result parameters are the same as the Result parameters returned by the model service that is used to evaluate the quality of a single video.
Video classification by label
You can classify short videos by adding labels to the videos. The mode service can return the class of the videos, the top K labels with the highest probabilities, and the probability of each label. The model service can also return the high-dimensional features of the videos. You can use one of the following methods to call the model service:
Classify a single video. The following code block shows a sample request:
from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi video_url = 'https://your_video_url.mp4' ai_service_api = AiServiceVideoApi(client) # The name of the custom model. model_name = '' # The title of the video. video_title = 'your_video_title' # The response settings of the model. configure = { 'output_classification': True, 'classification_top_k': 1, 'output_tag': True, 'tag_top_k': 5, 'output_embedding': True } # The model service that you want to call. response = ai_service_api.classify_label_video(video_url, video_title, model_name, configure) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The detailed status information about the request. message = response.message # The content returned by the model service. data = response.data # The ID of the job. job_id = response.data['job_id'] # Display the return value. print(response)
The following table describes the parameters in the preceding statement.
Parameter
Type
Required
Description
video_url
STRING
Yes
The URL of the video. We recommend that the size of the video is no larger than 100 MB and the resolution of the video is equal to or higher than 320 × 240.
video_title
STRING
Yes
The title of the video. If the video does not have a title, specify an empty string.
model_name
STRING
Yes
By default, an empty string is specified to call a general-purpose model.
If the general-purpose model cannot meet your requirements, you can contact the technical support of PAI to obtain a custom model. You can specify the name of a custom model to call the custom model.
configure
Dict
Yes
The response settings of the model. The default value is
None
. For more information, see Configure parameters.NoteIf configure is set to None, the parameters in the configure list use the default settings.
Table 6. Configure parameters
Parameter
Type
Required
Description
output_classification
BOOL
No
Specify whether to return the video classification result. Valid values:
True (default)
False
classification_top_k
INT
No
The top K classes of the video. Valid values of K: 1 to 10. Default value: 1.
output_tag
BOOL
No
Specify whether to return the labels of the video. Valid values:
True (default)
False
tag_top_k
INT
No
The top K labels of the video. Valid values of K: 1 to 10. Default value: 5.
output_embedding
BOOL
No
Specify whether to return the 64-dimensional video embedding. Valid values:
True (default)
False
Sample output:
{'code': 'OK', 'data': {'job_id': 5}, 'message': 'success', 'request_id': '4f6caecb-e3d6-4997-89fe-26ac7536****'}
Obtain job_id from the preceding returned response and specify the job ID in the following code to query the result.
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi ai_service_job_api = AiServiceJobApi(client) # Query the result. result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>) # Display the return value. print(result)
Replace <YOUR-JOB-ID> with the value of job_id returned by the model service, which is of the INT type.
The following result is returned if the job is running:
{'code': 'OK', 'data': {'job': {'Result': '', 'app_id': '202209070402410****', 'create_time': '2022-09-07T14:03:33.8+08:00', 'id': 5, 'message': 'video downloading', 'state': 1}}, 'message': 'success', 'request_id': '5259ec57-e55c-4772-b01e-3deafd74****'}
The following result is returned if the job is completed. Take note that some parameters contain untranslated Chinese text due to technical reasons.
{'code': 'OK', 'data': {'job': {'Result': '{"classification_result":[ {"class":"三农","score":0.997} ], "duration":14.633333333333333, "embedding_result":"-0.162,0.483,0.881,0.686,0.319,-1.569,0.266,0.227,-1.352,1.157,-0.784,-1.016,-0.987,-1.214,-0.493,-1.182,0.85,1.473,-1.27,-3.391,3.394,-0.471,-0.434,0.256,2.146,0.787,0.348,1.67,0.229,-0.878,0.299,0.392,-2.161,-0.756,-3.438,-0.158,0.713,-5.812,-1.248,0.253,-1.111,-0.165,2.644,-2.368,-1.636,-0.533,0.468,1.2,-1.431,-1.151,-1.276,-1.954,1.167,-0.938,-0.111,-2.636,-1.248,4.662,1.892,5.485,-1.137,1.589,-1.935,1.018", "tag_result":[ {"score":0.728,"tag":"盆栽"}, {"score":0.576,"tag":"植物"}, {"score":0.539,"tag":"种植技术"}, {"score":0.466,"tag":"农业技术"}, {"score":0.458,"tag":"种植"} ], "video_height":1280, "video_width":720}\n', 'app_id': '202209070402410****', 'create_time': '2022-09-07T14:03:33.8+08:00', 'id': 5, 'message': 'success', 'state': 2}}, 'message': 'success', 'request_id': '9283066a-f414-40fd-8fd3-0f77c1ef****'}
The following table describes the response parameters.
Parameter
Description
request_id
The request ID, which is of the STRING type.
code
The request status code, which is of the STRING type.
message
The detailed status information about the request, which is of the STRING type. Sample output:
success
Invalid Input - video data error - load video failed
Invalid Input - json format error - json parse runtime exception
For more information, see Error codes.
data
The content returned by the model service, which is of the DICT type. For more information, see Data ['job'] parameters.
Table 7. Data ['job'] parameters
Parameter
Description
id
The ID of the job (job_id), which is of the INT type.
app_id
The AppId, which is of the INT type.
state
The status code of the job, which is of the INT type. Valid values:
0: The job is initializing.
1: The job is running.
2: The job is completed.
3: The job failed.
message
The execution information of the job, which is of the STRING type. Valid values:
success
video downloading
create_time
The time when the job was created, which is of the STRING type.
Result
The result returned by the model service, which is of the STRING type. For more information, see Result parameters.
Table 8. Result parameters
Parameter
Description
classification_result
The classification result, which is of the LIST type. The classes are sorted in descending order by score (probability).
The length of the list varies based on classification_top_k.
This parameter is returned only when output_classification is set to True.
tag_result
The labels of the video, which are sorted in descending order by score (probability).
The length of the list varies based on tag_top_k.
This parameter is returned only when output_tag is set to True.
score
The probability of the label, which is of the FLOAT type. The value is accurate to three decimal places.
class
The alias of the class, which is of the STRING type.
tag
The name of the label, which is of the STRING type.
embedding_result
The 64-dimensional features of the video, which are of the STRING type. The features are separated by commas (,).
This parameter is returned only when output_embedding is set to True.
video_height
The height of the video in pixels, which is of the INT type.
video_width
The width of the video in pixels, which is of the INT type.
duration
The duration of the video, which is of the FLOAT type.
Classify multiple videos in a batch. The following code shows a sample request:
from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi from ai_service_python_sdk.client.models.batch_video_request_videos import BatchVideoRequestVideos ai_service_api = AiServiceVideoApi(client) # The response settings of the model. configure = { 'output_classification': True, 'classification_top_k': 1, 'output_tag': True, 'tag_top_k': 5, 'output_embedding': True } # Create a batch processing request (video URL + video title). videos = [ BatchVideoRequestVideos('https://your_video_1_url.mp4', 'your_video_title_1'), BatchVideoRequestVideos('https://your_video_2_url.mp4', 'your_video_title_2') ] # The name of the custom model. model_name = '' # The model service that you want to call. response = ai_service_api.batch_classify_label_video(videos, model_name, configure) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The detailed status information about the request. message = response.message # The content returned by the model service. data = response.data # batch id batch_id = response.data['batch_id'] # Display the return value. print(response)
Sample output:
{'code': 'OK', 'data': {'batch_id': 1}, 'message': 'success', 'request_id': 'c59eb912-9df4-4afe-8129-23f7b6dc****'}
Obtain batch_id from the preceding returned response and use the batch ID in the following code to query the result:
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi ai_service_job_api = AiServiceJobApi(client) # Query the result. result = ai_service_job_api.get_batch_job_with_id(<YOUR-BATCH-ID>) # Display the return value. print(result)
Replace <YOUR-BATCH-ID> with the value of batch_id returned by the model service, which is of the INT type.
The following result is returned. Take note that some parameters contain untranslated Chinese text due to technical reasons.
{'code': 'OK', 'data': {'jobs': [{'Result': '{"classification_result":[{"class":"生活","score":0.998}],"duration":14.633333333333333,"embedding_result":"0.998,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.002","tag_result":[{"score":0.917,"tag":"盆栽"},{"score":0.777,"tag":"植物"},{"score":0.693,"tag":"园艺"},{"score":0.58,"tag":"幸福树"},{"score":0.509,"tag":"有字幕"}],"video_height":1280,"video_width":720}', 'app_id': '202209290847310****', 'batch_id': 13, 'create_time': '2022-10-26T18:13:20.928+08:00', 'id': 4090, 'message': 'success', 'object': 'https://your_video_1_url.mp4', 'state': 2}, {'Result': '{"classification_result":[{"class":"体育","score":0.998}],"duration":18.2,"embedding_result":"0.0,0.0,0.0,0.998,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.001,0.0,0.0,0.0,0.0,0.001,0.0,0.0,0.0,0.0","tag_result":[{"score":0.981,"tag":"全运会"},{"score":0.901,"tag":"乒乓球"},{"score":0.798,"tag":"运动"},{"score":0.583,"tag":"有字幕"},{"score":0.498,"tag":"乒乓球比赛"}],"video_height":568,"video_width":320}', 'app_id': '202209290847310****', 'batch_id': 13, 'create_time': '2022-10-26T18:13:20.928+08:00', 'id': 4091, 'message': 'success', 'object': 'https://your_video_2_url.mp4', 'state': 2}]}, 'message': 'success', 'request_id': '73e4b48e-42b8-4347-9cb4-7a49c1f8****'}
The data['jobs'] parameter returned by the batch job is an array. The following table describes the parameters in the array.
Parameter
Description
id
The ID of the data entry in the batch job, which is of the INT type.
batch_id
The ID of the batch job, which is of the INT type.
app_id
The AppId, which is of the INT type.
state
The status code of the job, which is of the INT type. Valid values:
0: The job is initializing.
1: The job is running.
2: The job is completed.
3: The job failed.
message
The execution information of the job, which is of the STRING type. Valid values:
success
video downloading
create_time
The time when the job was created, which is of the STRING type.
object
The URL of the video, which is of the STRING type.
Result
The result returned by the model service, which is of the STRING type.
NoteThe Result parameters are the same as the Result parameters returned by the model service that is used to evaluate the quality of a single video.
Error codes
The following table describes the error codes returned by multimedia analysis SDK for Python.
Image analysis services
Request errors
HTTP status code | code | message | Operation |
400 | PARAMETER_ERROR |
| No image is specified in the request. |
| The AppId is invalid. | ||
| The data type of the image parameter is invalid. | ||
401 | PARAMETER_ERROR |
| The token is invalid. |
404 | PARAMETER_ERROR |
| The requested model service is not deployed. |
422 | SEVER_ERROR | Refer to the returned error message. | The model service encounters an error. Refer to the content of the returned message. |
Video analysis services
Request errors
HTTP status code
code
message
Operation
400
PARAMETER_ERROR
not found appid
The AppId is invalid.
401
PARAMETER_ERROR
sign error
The token is invalid.
404
PARAMETER_ERROR
model not found
The requested model service is not deployed.
Result query errors
state
message
Operation
3
failed to download video, err=fetch resource failed\tcode = 404
The system failed to download the video from the specified URL.
3
Refer to the returned error message.
The model service encounters an error. Refer to the content of the returned message.