ApsaraVideo Live:StartLiveMPUTask

Layout and volume SEI

Parameter	Description
canvas	The information about the canvas. The following fields are included:- w: the width of the canvas. Unit: pixels.- h: the height of the canvas. Unit: pixels.- bgnd: the background color of the canvas, which is an RGB value in the format of a hexadecimal integer.
stream	The information about the video stream. The following fields are included:- uid: the user ID of the streamer.- paneid: the sequence number of the pane on the canvas. Valid values: [0,8].- zorder: the layer of the pane. Valid values: [0,99].- x: the x-coordinate of the pane on the canvas. The value is normalized.- y: the y-coordinate of the pane on the canvas. The value is normalized.- w: the width of the pane. The value is normalized.- h: the height of the pane. The value is normalized.- type: the source of the video stream in the pane. Valid values: 0 (camera) and 1 (screen sharing).- status: the status of the video stream in the pane. Valid values: 0 (not pulled) and 1 (pulled).- muted: the mute status of the streamer. Valid values: 0 (not muted) and 1 (muted). For example, when two streamers (Streamer A and Streamer B) have a battle, if Streamer A mutes Streamer B, the muted field displayed for Streamer B is 1.- vol: the volume of the streamer. Valid values: [0,255]. Unit: dB.- vad: detects human voice. Valid values: [0,150]. 150 indicates that human voice is detected and a value other than 150 indicates the period of time during which the volume of human voice decreases to 0.
ts	The operating system timestamp when the SEI is generated. Unit: milliseconds.
ver	The version of the SEI. For example, the current version is 1.0.0.20220915.
udd	The custom scenario-based event that is sent by using the PassThrough parameter. The content of the event is specified by the PayloadContent parameter.

The following example shows a co-streaming scenario:

If a single streamer is streaming, the SEI message that viewers receive contains information about only one participant. If co-streaming or a battle occurs, the SEI message that viewers receive contains information about multiple participants. For example, when the streamer whose user ID is 111 is streaming, an SEI frame in the following format is sent to the viewer side:
{"canvas":{"w":1920,"h":1080,"bgnd":0},"stream":[{"uid":"111","paneid":-1,"zorder":0,"x":0,"y":0,"w":0,"h":0,"type":0,"status":1,"muted":0,"vol":0,"vad":0}],"ver":"1.0.0.20220915","ts":1697696105170} When the streamer whose user ID is 111 is co-streaming with a co-streamer whose user ID is 222, an SEI frame in the following format is sent to the viewer side:
{"canvas":{"w":1920,"h":1080,"bgnd":0},"stream":[{"uid":"111","paneid":0,"zorder":1,"x":0,"y":0.25,"w":0.5,"h":0.5,"type":0,"status":1,"muted":0,"vol":1,"vad":119},{"uid":"222","paneid":1,"zorder":1,"x":0.5018382,"y":0.25,"w":0.5,"h":0.5,"type":0,"status":1,"muted":0,"vol":60,"vad":123}],"ver":"1.0.0.20220915","ts":1697696106230} Viewers can determine whether a streaming layout changes based on the number of stream arrays. If only one stream array exists, a single streamer is streaming. If more than one stream array exists, co-streaming or a battle occurs. The layout information of the participants can indicate the position of each participant in the mixed stream.

SEI pass-through

• To configure custom SEI, call the StartLiveMPUTask operation to start a mixed-stream relay task and specify the PayloadContent parameter under the PassThrough parameter, or call the UpdateLiveMPUTask operation to update a mixed-stream relay task and specify the PayloadContent parameter under the PassThrough parameter.
• Custom SEI can be sent periodically. You can use the Interval parameter under the PassThrough parameter to specify the interval in milliseconds.
• Custom SEI can also be sent with keyframes by specifying the FollowIdr parameter under the PassThrough parameter.
  • You can configure settings to periodically send the custom SEI and send the custom SEI with keyframes at the same time. For example, you can set the Interval parameter to 1000 and the FollowIdr parameter to 1 to send the custom SEI every 1,000 milliseconds and include the custom SEI in keyframes.
  • If you do not specify the Interval parameter and the FollowIdr parameter, the custom SEI is sent only once when you call the operation.

For example, the streamer whose user ID is 111 ingests a stream and calls the UpdateLiveMPUTask operation to specify periodic SEI. In the operation, the Interval parameter is set to 1000, the FollowIdr parameter is set to 0, and the PayloadContent parameter is set to "hello world". In this case, the custom SEI is sent every 1,000 milliseconds. The SEI frame received on the viewer side is in the following format:
{"canvas":{"w":1920,"h":1080,"bgnd":0},"stream":[{"uid":"111","paneid":-1,"zorder":0,"x":0,"y":0,"w":0,"h":0,"type":0,"status":1,"muted":0,"vol":0,"vad":0}],"ver":"1.0.0.20220915","ts":1697696109876,"udd":"hello world"}

Mixed-stream relay across channels

If you want to mix streams from multiple streamers across multiple channels and relay the mixed stream to ApsaraVideo Live, you must specify the user ID and channel ID of the streamer who initiates the cross-channel call and the user IDs of other streamers as input parameters to create a mixed-stream relay task. Example: In a battle scenario, Streamer A in Channel A initiates a cross-channel battle with Streamer B in Channel B by using the client interface and relays the mixed stream to viewers in Channel A. In this example, the following parameters are specified:

• ChannelID: channelA
• UserId under UserInfos: userA and userB

Note To create a cross-channel mixed-stream relay task, make sure that a cross-channel call has been initiated by using the SDK on the client side. If users in different channels are not making a call with each other, you cannot create a cross-channel mixed-stream relay task. For more information about how to initiate a cross-channel call, see Cross-channel subscription.