Alibaba Cloud Push SDK is a development tool used to ingest streams from clients. This SDK is based on the powerful CDN and audio-video real-time communication technologies of Alibaba Cloud. It provides easy-to-use API operations, smooth and network-adaptive playback experience, multi-node-based latency optimization, effective real-time retouching, and other audio-video live streaming services. This topic provides an overview of Push SDK and describes the features, benefits, scenarios, and workflow of Push SDK.
Introduction
Alibaba Cloud Push SDK is a development tool used to ingest streams from clients. This SDK is based on the powerful CDN and audio-video real-time communication technologies of Alibaba Cloud. It provides easy-to-use API operations, smooth and network-adaptive playback experience, multi-node-based latency optimization, effective real-time retouching, and other audio-video live streaming services. Push SDK is free of charge. It reduces architectural design complexity and maintenance costs, which allows you to focus on improving your business logic implementation and user experience. For more information, visit the Push SDK page.
Push SDK integrates the smart retouching feature of Queen SDK and provides advanced retouching effects that are based on facial recognition. For example, you can use Push SDK to implement face slimming, face resizing, eye enlarging, and face whitening based on facial recognition.
To use the smart retouching feature, you must apply for a license. For more information about smart retouching and how to enable the feature, see Queen SDK overview.
Features
Feature | Description |
Stream ingest over Real-Time Messaging Protocol (RTMP) | Supports stream ingest over RTMP, and stream pulling over RTMP, Flash Video (FLV), HTTP Live Streaming (HLS), and Alibaba Real-Time Communication (ARTC). Supports resolutions from 180p to 720p. We recommend that you set the resolution to 540p. |
Web Real-Time Communications (WebRTC) stream ingest | Supports stream ingest over ARTC based on UDP. |
Co-streaming | Supports RTC-based co-streaming. |
Live stream recording | Push SDK for iOS: allows you to record live streams by using ReplayKit. Push SDK for Android: allows you to record mixed live streams from cameras. Supports Narrowband HD, which provides better image quality at the same bitrate. |
Supplemental enhancement information (SEI) insertion | Allows you to insert SEI to live streams. You can use players to parse SEI to implement features that are required by your business. |
Animated watermarks | Allows you to add or remove watermarks with animated effects to or from live streams in real time. |
Ingest of external audio and video streams | Allows you to ingest external audio and video streams for live streaming. |
Ingest of images | Allows you to ingest an image after you switch to the background or when the network connection is poor. |
Audio and video encoding | Supports software and hardware encoding of H.264 video files and Advanced Audio Coding (AAC) audio files. |
Real-time retouching | Supports advanced retouching effects based on facial recognition, including skin smoothing, skin brightening, face slimming, face resizing, and eye enlarging. |
Adaptive bitrate | Supports automatic bitrate adjustment during stream ingest based on network conditions. Multiple modes are supported to ensure smooth live streaming. |
Adaptive resolution | Supports automatic resolution adjustment during stream ingest based on network conditions. This adjustment is supported only by the quality-first mode and the smoothness-first mode. |
Background stream ingest | Ensures continuous stream ingest during the switch between the foreground and background. When you switch to the foreground, live streaming continues. |
Stereo stream ingest | Supports stereo stream ingest in mono or binaural mode. |
Multiple watermarks | Allows you to add up to three watermarks. You can change the position and size of the watermarks. |
Stream ingest in landscape mode | Supports stream ingest in portrait, landscape left, and landscape right modes. |
Configuration of stream collection parameters | Supports multiple stream collection parameters, such as the resolution, frame rate, audio sampling rate, group of pictures (GOP) size, and bitrate. You can configure these parameters to meet various stream collection requirements. |
Stream ingest in mirroring mode | Allows you to mirror the video images that are collected from cameras and ingested streams. The mirroring feature must be enabled for front cameras by default. |
Audio-only stream ingest | Supports the collection and ingest of only audio streams to save bandwidth and traffic in audio-only scenarios. |
Muted stream ingest | Allows you to mute the microphone and ingest only video images. |
Autofocus | Allows you to enable or disable the autofocus feature. You can also manually set the focus point. |
Image scaling | Allows you to scale the collected images based on the maximum zoom ratio supported by the camera. |
Camera switching and flash | Allows you to switch between the front camera and the rear camera, and turn on or turn off the flash when you use the rear camera. |
Background music | Supports background music. You can start, stop, pause, resume, or loop the playback of background music. |
Audio mixing | Allows you to mix music with vocals and adjust the volume of music and vocals. |
In-ear monitoring | Supports in-ear monitoring. For example, when streamers are singing with a headset, they can hear their voice from the headset in real time. This meets the live streaming requirements in karaoke scenarios. |
Noise reduction | Reduces noise that comes from the environment and mobile phones. |
Benefits
Ease of integration
Push SDK provides unified error codes and API operations that work in synchronous or asynchronous manners for Android and iOS apps. This meets integration requirements in different development architectures. Push SDK also provides comprehensive API references and demos for your information.
All-in-one solution
Push SDK delivers an all-in-one live streaming solution that integrates video stream collection, rendering, stream ingest, transcoding, distribution, and playback. Push SDK streamlines adaptive-bitrate stream ingest on clients, Narrowband HD™ transcoding in the cloud, and instant loading on clients to provide one-stop quality services.
High performance and low latency
Push SDK provides industry-leading solutions regarding the stuttering rate of stream ingest, CPU and memory consumption, power consumption, and heat generation. More than 2,500 points of presence (POPs) around the world ensure low latency.
WebRTC stream ingest
Push SDK provides WebRTC stream ingest based on UDP. You can add domain names for WebRTC stream ingest in the console as needed. WebRTC stream ingest can better prevent stuttering in poor upstream networks.
Co-streaming
Push SDK provides RTC-based co-streaming solutions. You can implement interactive streaming that engages more people at an ultra-low latency.
Scenarios
Scenario 1: Education live streaming
Description: Education live streaming focuses on the interaction between students and teachers. After students and teachers enter a live classroom, they can use Simple Message Queue (formerly MNS) to interact with each other in real time. Push SDK allows teachers to answer questions raised by students anytime and anywhere, in a timely and effective manner. Push SDK provides cloud-based recording and transcoding features. Therefore, students can replay class video recordings at any time to review the knowledge and enhance their learning.
Usage notes: Activate ApsaraVideo Live and enable the recording and transcoding features. Integrate Push SDK and Simple Message Queue (formerly MNS) SDK to implement the live streaming class and question-and-answer (Q&A) services. After you integrate ApsaraVideo Player SDK, students can watch live streams or replay recordings on the client to enjoy low-latency and interactive live streaming.
Scenario 2: Entertainment live streaming
Description: Due to the convenience of mobile phones, entertainment live streaming is a growing trend among users. Streamers require retouching and filter effects during live streaming. In addition, streamers interact with viewers by means of real-time chatting, liking, and rewarding activities to improve their popularity and program playability. However, the content must be strictly regulated to eliminate pornography or terrorism because entertainment live streaming is easy to access on mobile phones. You can use the live stream moderation feature to reduce review costs.
Usage notes: Activate ApsaraVideo Live and enable the recording and pornography detection features. Integrate Push SDK and enable the retouching feature to implement stream ingest. Integrate Simple Message Queue (formerly MNS) SDK of Alibaba Cloud to implement interactive chatting. This allows viewers to send text messages, voice messages, emojis, and images in the chat panel during live streaming. You can also build a rewarding system and use instant messaging to support payment. After you integrate ApsaraVideo Player SDK, viewers can watch live streams or replay recordings on the client.
Scenario 3: Game live streaming
Description: To implement game live streaming on mobile phones, use the screen recording technology to merge the game stream and the stream that is collected by the camera. Then, use Push SDK to ingest the merged streams. Screen recording must be enabled when you use Push SDK to support game live streaming. The interaction between streamers and viewers in game live streaming is similar to that in entertainment live streaming. You can use Simple Message Queue (formerly MNS) SDK of Alibaba Cloud to implement interaction features, such as chatting, liking, and rewarding. You can also use the live stream recording feature to replay clips.
Usage notes: Activate ApsaraVideo Live and enable live recording. Integrate Push SDK and enable the live recording feature to implement stream ingest. Integrate Simple Message Queue (formerly MNS) SDK of Alibaba Cloud to implement interactive chatting. This allows viewers to send messages, emojis, and images in the chat panel during live streaming. You can also build a rewording system and use instant messaging to support payment. Integrate ApsaraVideo Player SDK to achieve instant loading and dynamic frame synchronization if you want viewers to watch or replay a live stream.
Scenario 4: Co-streaming
Description: Push SDK can provide real-time interactive audio and video services that are stable and have high quality and ultra-low latency. By using the co-streaming feature, a streamer can co-stream with other streamers or viewers, and streamers across rooms can interact with each other. Multiple screens are displayed, and the video streams are ingested to live centers for large-scale distribution to other viewers.
Usage notes: Enable the co-streaming feature in the ApsaraVideo Live console and integrate the interactive edition of Push SDK into your app. This way, users can use the co-streaming feature to make audio and video calls with other users in real time. This feature is suitable for mobile social networking scenarios.
Workflow of Push SDK
Workflow of the basic edition of Push SDK
Your app initiates a request to the AppServer to obtain an ingest URL.
The AppServer constructs the ingest URL based on the concatenation rules and returns the ingest URL to your app.
Your app passes the ingest URL to Push SDK and uses Push SDK to start stream ingest.
Push SDK ingests live streams to CDN.
Workflow of the interactive edition of Push SDK
A streamer and co-streamer initiate a request to the AppServer to obtain an ingest URL.
The AppServer constructs the ingest URL based on the concatenation rules and returns the ingest URL to your app.
The streamer and co-streamer pass the ingest URL to the Alibaba Cloud server to start stream ingest. An interactive stream with an ultra-low latency is obtained.
Alibaba Cloud uses stream relay and stream mixing to generate streams in various formats and ingests them to CDN.
Viewers initiate a request to the AppServer to obtain a streaming URL.
The AppServer constructs the streaming URL based on the concatenation rules and returns the streaming URL to your app.
Viewers use the streaming URL to pull streams from CDN.
Development support
If you have any questions or suggestions about Push SDK, join the developer ecosystem group of Push SDK by searching for group number 32825314 on DingTalk, or join the developer group of the all-in-one SDK by searching for group number 44911608 on DingTalk.