You can call the Live Stream Moderation 2.0 API to identify risky content in the images and voices in live streams. This topic describes how to call the Live Stream Moderation 2.0 API to moderate live streams.
Usage instructions
Create an Alibaba Cloud account: Click Register account and follow the instructions to create an Alibaba Cloud account.
Activate the pay-as-you-go billing method for Content Moderation: Make sure that the Content Moderation 2.0 service is activated. You are not charged for activating this service. After you call API operations, the billing system automatically charges you based on your usage. For more information, see Billing methods.
Create an AccessKey pair: Make sure that you have created an AccessKey pair as a Resource Access Management (RAM) user. For more information, see Create an AccessKey pair. If you want to use the AccessKey pair of the RAM user, use your Alibaba Cloud account to grant the AliyunYundunGreenWebFullAccess permission to the RAM user. For more information, see Grant permissions to a RAM user.
Use SDKs: We recommend that you use SDKs to call API operations. For more information, see Guide of using Video Moderation 2.0.
The Live Stream Moderation service provides the following three API operations:
VideoModeration: for submitting a live stream moderation task
VideoModerationResult: for obtaining the moderation results of a live stream moderation task
VideoModerationCancel: for canceling a live stream moderation task
Submit a moderation task
Usage notes
Operation: VideoModeration (only for asynchronous moderation)
Supported regions and endpoints
Region
Public endpoint
Internal endpoint
Supported service
Singapore
green-cip.ap-southeast-1.aliyuncs.com
green-cip-vpc.ap-southeast-1.aliyuncs.com
liveStreamDetection_global
Billing methods
You are charged for calling this operation. The billing system charges you based on the image moderation policy and voice moderation policy that you set for live streams. If multiple moderation services are called to moderate the images in live streams, fees are accumulated based on the number of captured frames multiplied by the unit price of each moderation service. If live streams are moderated for voice content violations at the same time, you are also charged for the video duration multiplied by the unit price of voice moderation. For more information about the billing method, see Billing methods.
Moderation objects: Live streams
Returned results: If you send asynchronous moderation requests, the moderation results are not returned in real time. To obtain moderation results, you can poll the moderation results periodically or enable callback notification. The moderation results are retained for up to 24 hours.
Enable callback notification to obtain moderation results: When you submit an asynchronous moderation task, you can specify a callback URL for receiving moderation results in the callback parameter of the moderation request.
Poll moderation results: You do not need to set the callback parameter when you submit asynchronous moderation tasks. After you submit the tasks, you can call the result query operation to query moderation results.
Limits on live streams
Live streams complying with the following protocols are supported: RTMP, HLS, HTTP-FLV, and RTSP.
Limits on the duration of live streams: The maximum duration of a single live stream that can be moderated is 24 hours. When the duration exceeds 24 hours, the moderation task automatically ends.
Moderation rule configurations
Before you call the Live Stream Moderation API for the first time, you must configure live stream moderation rules in the Content Moderation console.
If you do not configure live stream moderation rules, the Live Stream Moderation 2.0 API uses the following default configurations:
Live stream moderation (liveStreamDetection_global)
Frequency of capturing frames: 1 second/frame
Image moderation service: common baseline moderation (baselineCheck_global)
Voice moderation: enabled
Voice moderation service: voice moderation (live_stream_detection_global)
Returned results: Only risky content is returned.
QPS limits
You can call this operation up to 100 times per second per account. Only 50 moderation tasks can be processed at a time. If you need to increase the number of concurrent moderation tasks, consult your business manager. If the number of calls per second exceeds the limit, throttling is triggered. As a result, your business may be affected. We recommend that you take note of the limit when you call this operation.
Debugging
Before you deploy SDKs, you can use Alibaba Cloud OpenAPI Explorer to debug the VideoModeration operation online and view the sample code for the call and SDK dependencies. This way, you can understand how to call the operation and how to set related parameters.
Before you call the Content Moderation API, you must log on to the Content Moderation console by using your Alibaba Cloud account. Therefore, the fees incurred by calling the operations are billed to the account.
Request parameters
Parameter | Type | Required | Example | Description |
Service | String | Yes | liveStreamDetection_global | The type of the moderation service. Valid values:
|
ServiceParameters | JSONString | Yes | The parameters required by the moderation service. The value is a JSON string. For more information about the description of each string, see Table 1 ServiceParameters. |
Table 1 ServiceParameters
Parameter | Type | Required | Example | Description |
url | String | Yes | http://www.aliyundoc.com/a.flv | The URL of the object that you want to moderate. Make sure that the URL can be accessed over the Internet and that the URL cannot exceed 2,048 characters in length. Note The URL cannot contain Chinese characters. Make sure that you specify only one URL in each request. |
callback | String | No | http://www.aliyundoc.com | The callback URL for notifying you of asynchronous moderation results. HTTP and HTTPS URLs are supported. If you do not set this parameter, you must poll moderation results periodically. If you set the callback parameter in the moderation request, make sure that the specified HTTP or HTTPS URL meets the following requirements: supports the POST method, uses UTF-8 to encode the transmitted data, and supports the checksum and content parameters. To send moderation results to the specified callback URL, Content Moderation returns the checksum and content parameters in callback notifications based on the following rules and format:
Note If your server receives a callback notification, the server sends an HTTP 200 status code to Content Moderation. If your server fails to receive a callback notification, the server sends other HTTP status codes to Content Moderation. If your server fails to receive a callback notification, Content Moderation continues to push the callback notification until your server receives it. Content Moderation can push a callback notification repeatedly up to 16 times. After 16 times, Content Moderation stops pushing the callback notification. In this case, we recommend that you check the status of the callback URL. |
seed | String | No | abc**** | A random string that is used to generate a signature for the callback notification request. The string can be up to 64 characters in length and can contain letters, digits, and underscores (_). You can customize this string. It is used to verify the callback notification request when Content Moderation pushes callback notifications to your server. Note This parameter is required if you set the callback parameter. |
cryptType | String | No | SHA256 | The algorithm used to sign the callback notification content when you enable callback notification. Content Moderation signs the returned string by using the algorithm that you specify and sends the signed string to the callback URL. The returned string is in the UID + Seed + Content format. Valid values:
|
dataId | String | No | videoId**** | The ID of the moderated object. The ID can contain letters, digits, underscores (_), hyphens (-), and periods (.). It can be up to 128 characters in length. This ID uniquely identifies your business data. |
liveId | String | No | liveId**** | The ID of the live stream. This parameter is used to prevent repeated moderation of live streams. If you specify this parameter, Content Moderation checks whether a task for moderating the specified live stream is in progress based on the ID of the Alibaba Cloud account, the moderation service, and the liveId parameter. If the moderation task is in progress, the taskId of the task is returned and no new task is initiated. |
If your server successfully receives a callback notification, the server sends an HTTP 200 status code to Content Moderation. If your server fails to receive a callback notification, the server sends other HTTP status codes to Content Moderation. If your server fails to receive a callback notification, Content Moderation continues to push the callback notification until your server receives it. Content Moderation can push a callback notification repeatedly up to 16 times. After 16 times, Content Moderation stops pushing the callback notification. In this case, we recommend that you check the status of the callback URL.
Response parameters
Parameter | Type | Example | Description | |
Code | Integer | 200 | The HTTP status code. For more information, see Response codes. | |
Data | JSONObject | {"TaskId": "AAAAA-BBBBB"} | The moderation results. | |
Message | String | OK | The message that is returned in response to the request. | |
RequestId | String | ABCD1234-1234-1234-1234-123**** | The request ID. | |
Examples
Sample requests
{
"Service": "liveStreamDetection_global",
"ServiceParameters": {
"url": "http://www.aliyundoc.com/a.flv",
"dataId": "videoId****",
"liveId": "liveId****"
}
}Sample success responses
{
"Msg": "OK",
"Code": 200,
"Data":
{
"TaskId": "AAAAA-BBBBB"
"DataId": "videoId****"
},
"RequestId": "ABCD1234-1234-1234-1234-123****"
}Obtain the moderation results of a live stream moderation task
Usage notes
Operation: VideoModerationResult
Billing: This operation is free of charge.
Response timeout: We recommend that you query moderation results at least 30s after you send an asynchronous moderation request. Content Moderation retains moderation results for up to 24 hours. After 24 hours, the results are deleted.
Returned results: If the live stream is in progress, the latest 10 moderation results are returned for each query. If the live stream has ended, all moderation results are returned.
QPS limits
You can call this operation up to 100 times per second per account. If the number of calls per second exceeds the limit, throttling is triggered. As a result, your business may be affected. We recommend that you take note of the limit when you call this operation.
Debugging
Before you deploy SDKs, you can use Alibaba Cloud OpenAPI Explorer to debug the VideoModerationResult operation online and view the sample code for the call and SDK dependencies. This way, you can understand how to call the operation and how to set related parameters.
Request parameters
Parameter | Type | Required | Example | Description |
Service | String | Yes | liveStreamDetection_global | The type of the moderation service. It must be the same as the type of the moderation service for the submitted moderation task. |
ServiceParameters | JSONString | Yes | The parameters required by the moderation service. The value is a JSON string. For more information about the description of each string, see Table 1 ServiceParameters. |
Table 1 ServiceParameters
Parameter | Type | Required | Example | Description |
taskId | string | Yes | abcd**** | The ID of the moderation task that you want to query. You can enter one task ID at a time. Note After you submit a moderation task, you can obtain the ID of the task from the response. |
Response parameters
Parameter | Type | Example | Description |
RequestId | String | ABCD1234-1234-1234-1234-123**** | The request ID, which is used to locate and troubleshoot issues. |
Data | Object | The moderation results of the live stream. For more information, see Table 2 Data. | |
Code | String | 200 | The HTTP status code. For more information, see Response codes. |
Message | String | OK | The message that is returned in the response. |
Table 2 Data
Parameter | Type | Example | Description |
DataId | String | videoId**** | The ID of the moderated object. Note If you specify the dataId parameter in the request, the value of the dataId parameter is returned in the response. |
LiveId | String | liveId**** | The live stream ID corresponding to the moderated object. Note If you specify the liveId parameter in the request, the value of the liveId parameter is returned in the response. |
TaskId | String | AAAAA-BBBBB | The task ID corresponding to the moderated object. |
RiskLevel | String | high | The risk level of a live stream. The risk level is determined based on a comprehensive assessment of both video images and the voice in a video. The return values include:
Note For content flagged as high risk, we recommend taking immediate disposition actions. For medium-risk content, manual review is advised. Low-risk content can be handled when there is a high recall requirement. Otherwise, it can be treated in the same manner as content with no risk detected in routine operations. The risk score for video images can be configured in Video Moderation Plus. |
FrameResult | JSONObject | The image moderation results. If the call is successful, the HTTP status code 200 and moderation results are returned. The moderation results contain a structure. For more information about the structure, see Table 3 FrameResult. Note In the scenario of moderating a live stream, the HTTP status code 280 indicates that the moderation task is in progress, and the HTTP status code 200 indicates that the moderation task is complete. If the moderation task is in progress, the returned moderation results contain all the issues that Content Moderation has detected in the task. | |
AudioResult | JSONObject | The voice moderation results. The moderation results contain a structure. For more information about the structure, see Table 8 audioResult. |
Table 3 FrameResult
Parameter | Type | Example | Description |
FrameNum | Integer | 200 | The number of captured frames that are returned for the live stream. |
FrameSummarys | JSONArray | The summary of the labels of the live stream against which captured frames are matched. For more information, see Table 4 FrameSummarys. | |
RiskLevel | String | high | The risk level of the screen of a live stream. The risk level is calculated and returned based on an analysis of all captured frames during the live stream. The return values include:
|
Frames | JSONArray | The information about the frames that match the labels. For more information, see Table 5 Frames. |
Table 4 FrameSummarys
Parameter | Type | Example | Description |
Label | String | violent_armedForces | The label against which captured frames is matched. |
Description | String | Suspected content about smoke or fire is contained. | The explanation for the Label field. Note This field is an explanatory note for the Label field. It is subject to change and adjustment. Therefore, we recommend making dispositions based on the Label field rather than relying solely on this explanatory field for final decision-making. |
LabelSum | Integer | 8 | The number of times that the label is matched. |
Table 5 Frames
Parameter | Type | Example | Description |
TempUrl | String | http://www.aliyundoc.com/test.jpg | The temporary URL of the captured frame of the live stream. This URL is valid for 30 minutes. We recommend that you store the captured frame to an Object Storage Service (OSS) bucket in a timely manner. Note If the feature of storing video evidence in OSS buckets is enabled, this parameter returns the OSS URL of the captured frame. |
Offset | Float | 50.5 | The interval between the start of the live stream and the captured frame. Unit: seconds. |
Timestamp | Integer | 1684559739000 | The absolute timestamp when the captured frame was processed. Unit: milliseconds. |
RiskLevel | String | high | The risk level of live stream frames. The risk level is determined based on the pre-set high and low risk scores. The return values include:
Note For content flagged as high risk, we recommend taking immediate disposition actions. For medium-risk content, manual review is advised. Low-risk content can be handled when there is a high recall requirement. Otherwise, it can be treated in the same manner as content with no risk detected in routine operations. The risk score for video frames can be configured in Video Moderation Plus. |
Results | JSONArray | The results of frame moderation parameters such as the label parameter and the confidence parameter. For more information, see Table 6 Results. |
Table 6 Results
Parameter | Type | Example | Description |
Service | String | baselineCheck_global | The image moderation service that is called. |
Result | Array | The results of frame moderation parameters such as the label parameter and the confidence parameter. For more information, see Table 7 Result. |
Table 7 Result
Parameter | Type | Example | Description |
Label | String | violent_explosion | The label returned after a frame is moderated. Multiple risk labels and the corresponding scores of confidence levels may be returned for a frame. Valid values: |
Confidence | Float | 81.22 | The score of the confidence level. Valid values: 0 to 100. The value is accurate to two decimal places. |
Description | String | Suspected content about smoke or fire is contained. | The explanation for the Label field. Note This field is an explanatory note for the Label field. It is subject to change and adjustment. Therefore, we recommend making dispositions based on the Label field rather than relying solely on this explanatory field for final decision-making. |
Table 8 audioResult
Parameter | Type | Example | Description |
AudioSummarys | JSONArray | Summary of voice labels. For more information about the structure, see Table 9 AudioSummarys. | |
RiskLevel | String | high | The voice risk level. The risk level is calculated and returned based on an analysis of all voice segments. The return values include:
|
SliceDetails | JSONArray | The details about the text in the moderated voice. The value is a JSON array that contains one or more elements. Each element corresponds to a text entry. For more information about the structure, see Table 10 SliceDetails. |
Table 9 AudioSummarys
Header | Type | Example | Description |
Label | String | profanity | Voice label. |
LabelSum | Integer | 8 | The number of times that the label is matched. |
Table 10 SliceDetails
Parameter | Type | Example | Description |
StartTime | Integer | 0 | The start time of the text after voice-to-text conversion. Unit: seconds. |
EndTime | Integer | 4065 | The end time of the text after voice-to-text conversion. Unit: seconds. |
StartTimestamp | Integer | 1678854649720 | The start timestamp of the segment. Unit: milliseconds. |
EndTimestamp | Integer | 1678854649720 | The end timestamp of the segment. Unit: milliseconds. |
Text | String | Disgusting | The text converted from voice. |
Url | String | https://aliyundoc.com/test.wav | If the moderation object is a voice stream, this parameter indicates the temporary access URL of the voice stream to which the text entry corresponds. The validity period of the URL is 30 minutes. You must prepare another URL to store the audio stream at the earliest opportunity. |
Labels | String | political_content,xxxx | The details of the labels. Multiple labels are separated by commas (,). Valid values:
|
RiskLevel | String | high | The risk level of voice segments in a video. The return values include:
|
RiskWords | String | AAA, BBB, CCC | The risk words that are hit. Multiple words are separated by commas (,). |
RiskTips | String | Pornography_vulgar words, pornography_description | Subcategory labels. Multiple labels are separated by commas (,). |
Extend | String | {"riskTips":"Pornography_vulgar words","riskWords":"Pornography service"} | A reserved parameter. |
Examples
Sample requests
{
"Service": "liveStreamDetection_global",
"ServiceParameters": {
"taskId": "abcd****"
}
}Sample success responses
Moderate only images in a live stream
{
"Code": 200,
"RequestId": "25106421-XXXX-XXXX-XXXX-15DA5AAAC546",
"Message": "success finished",
"Data": {
"DataId": "dc16c28f-xxxx-xxxx-xxxx-51efe0131080",
"LiveId": "live0307-0728-****",
"RiskLevel": "high",
"FrameResult": {
"FrameNum": 2,
"FrameSummarys": [
{
"Label": "violent_explosion",
"Description": "Suspected content about smoke or fire is detected",
"LabelSum": 8
},
{
"Label": "sexual_cleavage",
"Description": "Suspected of containing nudity or sexy content involving body exposure",
"LabelSum": 8
}
],
"RiskLevel": "medium",
"Frames": [
{
"Offset": 1,
"RiskLevel": "none",
"Results": [
{
"Result": [
{
"Label": "nonLabel",
"Description": "No risk detectd"
}
],
"Service": "baselineCheck_global"
}
],
"TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test1.jpg",
"Timestamp": 1704769999000
},
{
"Offset": 2,
"RiskLevel": "medium",
"Results": [
{
"Result": [
{
"Confidence": 78.88,
"Label": "sexual_cleavage",
"Description": "Suspected of containing nudity or sexy content involving body exposure"
},
{
"Confidence": 74.18,
"Label": "violent_explosion",
"Description": "Suspected content about smoke or fire is detected"
}
],
"Service": "baselineCheck_global"
}
],
"TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test2.jpg",
"Timestamp": 1704769989000
}
]
}
}
}Moderate both images and voices in a live stream
{
"Code": 200,
"RequestId": "25106421-XXXX-XXXX-XXXX-15DA5AAAC546",
"Message": "success finished",
"Data": {
"DataId": "dc16c28f-xxxx-xxxx-xxxx-51efe0131080",
"LiveId": "live0307-0728-****",
"RiskLevel": "high",
"AudioResult": {
"AudioSummarys": [
{
"Label": "sexual_sounds",
"LabelSum": 3
}
],
"RiskLevel": "high",
"SliceDetails": [
{
"EndTime": 60,
"EndTimestamp": 1698912813192,
"Labels": "",
"RiskLevel": "none",
"StartTime": 30,
"StartTimestamp": 1698912783192,
"Text": "Content Moderation",
"Url": "http://abc.oss-ap-southeast-1.aliyuncs.com/test.wav"
},
{
"EndTime": 30,
"EndTimestamp": 1698912813192,
"Extend": "{\"customizedWords\":\"service\",\"customizedLibs\":\"test\"}",
"Labels": "C_customized",
"RiskLevel": "high",
"StartTime": 0,
"StartTimestamp": 1698912783192,
"Text": "Welcome to the Alibaba Cloud Content Moderation servic",
"Url": "http://abc.oss-ap-southeast-1.aliyuncs.com/test.wav"
}
]
},
"FrameResult": {
"FrameNum": 2,
"FrameSummarys": [
{
"Label": "violent_explosion",
"Description": "Suspected content about smoke or fire is detected",
"LabelSum": 8
},
{
"Label": "sexual_cleavage",
"Description": "Suspected of containing nudity or sexy content involving body exposure",
"LabelSum": 8
}
],
"RiskLevel": "medium",
"Frames": [
{
"Offset": 1,
"RiskLevel": "none",
"Results": [
{
"Result": [
{
"Label": "nonLabel",
"Description": "No risk detectd"
}
],
"Service": "baselineCheck_global"
}
],
"TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test1.jpg",
"Timestamp": 1704769999000
},
{
"Offset": 2,
"RiskLevel": "medium",
"Results": [
{
"Result": [
{
"Confidence": 74.18,
"Label": "violent_explosion",
"Description": "v"
},
{
"Confidence": 78.88,
"Label": "sexual_cleavage",
"Description": "Suspected of containing nudity or sexy content involving body exposure"
}
],
"Service": "baselineCheck_global"
}
],
"TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test2.jpg",
"Timestamp": 1704769989000
}
]
}
}
}Cancel a live stream moderation task
Usage notes
You can call this operation to cancel a live stream moderation task.
Service: VideoModerationCancel
Billing: This operation is free of charge.
Debugging
Before you deploy SDKs, you can use Alibaba Cloud OpenAPI Explorer to debug the VideoModerationCancel operation online and view the sample code for the call and SDK dependencies. This way, you can understand how to call the operation and how to set related parameters.
Request parameters
Parameter | Type | Required | Example | Description |
Service | String | Yes | liveStreamDetection_global | The type of the moderation service. It must be the same as the type of the moderation service for the submitted moderation task. |
ServiceParameters | JSONString | Yes | The parameters required by the moderation service. The value is a JSON string. For more information about the description of each string, see Table 11 ServiceParameters. |
Table 11 ServiceParameters
Parameter | Type | Required | Example | Description |
taskId | string | Yes | abcd**** | The ID of the moderation task that you want to cancel. You can enter one task ID at a time. Note: After you submit a moderation task, you can obtain the ID of the task from the response. |
Response parameters
Parameter | Type | Example | Description |
Code | Integer | 200 | The HTTP status code. For more information, see Response codes. |
Message | String | OK | The message that is returned in the response. |
RequestId | String | AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE**** | The ID of the request. |
Examples
Sample requests
{
"Service": "liveStreamDetection",
"ServiceParameters": {
"taskId": "abcd****"
}
}Sample responses
{
"Msg": "OK",
"Code": 200,
"RequestId": "70ED13B0-BC22-576D-9CCF-1CC12FEAC477"
}Response codes
The following table describes the response codes for calling Live Stream Moderation 2.0 API operations. You are charged by using the pay-as-you-go billing method only for requests whose response codes are 200 and 280.
Code | Description |
200 | The request is normal or the moderation is complete. |
280 | The moderation is in progress. |
400 | Not all request parameters are configured. |
401 | The values specified for one or more request parameters are invalid. |
402 | Invalid value length of request parameters. Check and modify them and try again. |
403 | The QPS of requests exceeds the upper limit. Check and modify the number of requests that are sent at a time. |
404 | The specified video file failed to be downloaded. Check the URL of the video file or try again. |
405 | Downloading the specified video file timed out. The possible cause is that the video file cannot be accessed. Check and adjust the video file and try again. |
406 | The specified video file is excessively large. Check and change the video file size and try again. |
407 | The format of the specified video file is not supported. Check and change the video file format and try again. |
408 | You do not have the required permissions. The possible cause is that this account is not activated, has overdue payments, or is not authorized to call this API. |
409 | The specified task ID does not exist. The possible cause is that the moderation results have exceeded the 24-hour validity period. |
480 | The number of concurrent moderation tasks exceeds the upper limit. Check and change the number of concurrent moderation tasks. |
500 | A system exception occurred. |