ApsaraVideo Live:Common causes of and solutions to high streaming latency

Common causes

Stream ingest

Recommended settings for OBS Studio

The following figure shows the recommended parameter settings in the stream ingest tool OBS Studio:

Recommended settings for vMix

The following figure shows the recommended parameter settings in the stream ingest tool vMix:

Optimal settings for Push SDK

In Push SDK, set the group of pictures (GOP) size to 1 and retain the default settings for other parameters.

mAlivcLivePushConfig = new AlivcLivePushConfig();
mAlivcLivePushConfig.setVideoEncodeGop(AlivcVideoEncodeGopEnum.GOP_ONE);

AlivcLivePushConfig *pushConfig = [[AlivcLivePushConfig alloc] init];
pushConfig.videoEncodeGop = AlivcLivePushVideoEncodeGOP_1

Check the network environment for stream ingest

Make sure that the network condition on the stream ingest end is good when you ingest a stream. If you are not sure whether the high streaming latency is caused by network issues, we recommend that you switch the network and use the same device to ingest the stream again. If the latency returns to normal after you perform the preceding operations, there may be a problem with the previous network.

Check the stream ingest device load

If your stream ingest device is overloaded, for example, the CPU or memory usage is too high, the efficiency of video capture and encoding/decoding is affected. This further affects the latency of live streaming. If you are sure that the stream ingest device is overloaded, replace the device in a timely manner. If you are not sure whether the stream ingest device is overloaded, we recommend that you keep the same network environment and stream ingest configurations and use another device for verification.

Playback

Optimal settings for Native ApsaraVideo Player SDK

RTS uses Web Real-Time Communication (WebRTC) to play streams, which provides a good resistance to packet loss and jitter on the playback end. You can further optimize the buffer control of the player to obtain a lower latency.

• If you integrate RTS SDK with players that are not developed by Alibaba Cloud, you need to adjust the buffer control policy. For example, if you use FFplay, run the following command to minimize the buffer of FFplay:

  $ ./ffplay -fflags nobuffer -flags low_delay artc://...

• If you use ApsaraVideo Player SDK V5.5.6.0 or later or use the demo of ApsaraVideo Player, the optimal settings already apply and no additional adjustment is required.

If the streaming latency is not reduced after you complete the preceding configurations, perform further analysis on RTS SDK:

• Analyze the streaming latency: Enable logging for ApsaraVideo Player SDK for Android or ApsaraVideo Player SDK for iOS, and query the end-to-end latency in the code=154 log (supported by RTS SDK 2.4.0 or later). The latency is measured in milliseconds. Take note of the following fields:

  • gd (globalDelayMS): the end-to-end latency.

  • td (transDelayMS): the stream ingest latency plus the CDN transmission latency. If the value is greater than 800 milliseconds, the stream ingest latency or CDN transmission latency is excessively large.

  • nd (network delay): the network latency from CDN POPs to the player. If the value is greater than 800 milliseconds, a network issue may exist.

  • jd (jitterDelayMS): the de-jitter buffer latency. If the value is greater than 800 milliseconds, a network issue may exist.

  • ud: the length of the user buffer. In most cases, the value is 0. You can ignore this field.

  • dd (decoder delay): the decoding latency from the player. If the value is greater than 800 milliseconds, the player may have a decoding issue.

  • rd (render delay): the rendering latency from the player. If the value is greater than 800 milliseconds, the player may have a rendering issue.

  • ut: You can ignore this field.

• Analyze the first frame latency: Query the following keywords in the code=161 log of RTS SDK.

  st:0,init:1,sdns:0,rdns:2,sc:0,ced:12,sub:5,frsp:309,si:8,fp:13,ffc:5,ffo:416,sum:763

  If the value of sum is significantly different from the first frame latency, the player has decoding and rendering issues. If the value of sum is equal to or close to the first frame latency, analyze the values of all fields. If the values of ced, frsp, and ffc are larger than usual, a network issue exists.

Optimal settings for ApsaraVideo Player SDK for Web

When you use ApsaraVideo Player SDK for RTS-based streaming, the buffer is controlled by the default policy of the browser. No additional configuration is required. If the streaming latency or the first frame latency is high when you use ApsaraVideo Player SDK for Web to play streams, you can use Google Chrome to visit chrome://webrtc-internals and check whether the WebRTC data that is being pulled is abnormal.

Take note of the following metrics:

• inbound-rtp (kind=audio) and inbound-rtp (kind=video): the real-time data receiving status and the frame decoding status, respectively. If the value of framesDropped is high or the value of framesDecoded/s is different from your expected bitrate, a decoding issue exists. Troubleshoot the stream ingest or transcoding stage.

• candidate-pair: the connection status. If the value of currentRoundTripTime is high or the value of packetsLost in inbound-rtp is high, a network issue may exist.

Latency that occurs when you play audio-only or video-only streams

By default, Alibaba Cloud waits for 5 seconds to obtain the complete audio and video data when playing an audio-only or video-only stream. As a result, a black screen of 5 seconds is displayed. You can manually unsubscribe from video or audio by adding @subvideo=no or @subaudio=no at the end of the streaming URL.

• If the source stream is a video-only stream for which the streaming URL is artc://example.aliyundoc.com/app/stream?auth_key={Access token}, modify the streaming URL to artc://example.aliyundoc.com/app/stream?auth_key={Access token}@subaudio=no.

• If the source stream is an audio-only stream for which the streaming URL is artc://example.aliyundoc.com/app/stream?auth_key={Access token}, modify the streaming URL to artc://example.aliyundoc.com/app/stream?auth_key={Access token}@subvideo=no.

Check the network environment for playback

Make sure that the network condition on the playback end is good when you play a stream. If your network is unstable, ApsaraVideo Player SDK automatically increases the size of the local buffer to provide smoother playback. As a result, the latency is increased.

Playback

If transcoding is performed during live streaming, the streaming latency can increase by 300 to 500 milliseconds. RTS provides the following types of transcoding:

• HTML5 auto transcoding: This type of transcoding is used when ingested RTMP streams are played on web pages. The latency caused by auto transcoding cannot be avoided. However, it is relatively low and does not have a big impact on the overall streaming latency.

• Custom transcoding: If you configure transcoding templates of multiple specifications for a live stream, the latency comes from up to two parts when you play the transcoded stream:

  • Latency from transcoding processing: This part of latency is estimated to be 300 to 500 milliseconds and cannot be avoided.

  • Latency from transcoding initiation: If triggered transcoding is enabled in your transcoding template, transcoding is initiated only when a user pulls the stream for the first time. This causes additional latency for users who pull the stream for the first time. This part of latency is estimated to be 200 milliseconds.

    Note

    Triggered transcoding is an optional feature that you can configure when you create a transcoding template. To disable triggered transcoding for a stream, you must create a new transcoding template in which triggered transcoding is disabled. Then, re-ingest the stream and test the streaming latency again.

Contact us

If the latency is still not reduced after you perform the preceding troubleshooting steps, submit a ticket. To improve communication efficiency, add the following information to the ticket:

• The ingest URL and streaming URL that you used for testing.

• The stream ingest and playback tools. Example: OBS Studio for stream ingest and ApsaraVideo Player SDK for Web for playback.

• RTS version: RTS 1.0 or RTS 2.0.

• Other useful information mentioned in the preceding troubleshooting steps.