The short video SDK provides upgraded recording features based on basic recording. In addition to all features of basic recording, the short video SDK also supports view recording, also called screen recording. You can combine videos that are collected from multiple sources, such as videos recorded in the view, videos recorded by using the camera, and local videos. This allows you to perform real-time recording and production in various scenarios. For example, you can combine videos recorded by using the camera and videos recorded in the view to teach students with an online whiteboard, or you can combine videos recorded by using the camera and local videos to perform duet recording.

Supported editions

Edition Supported
Professional Yes
Standard Yes
Basic No

Terms

This section describes the terms that help you better understand multi-source recording. For more information, see Multi-source recording, Track, and Track layout.

Related classes

Class Description
AliyunRecorder A core class that defines multi-source recording features, including recording, preview settings, effect settings, and callback settings.
AliyunRecorderConfig A class that defines multi-source recording parameters, including the output path, video sources, watermarks, and background music.
AliyunRecorderVideoConfig A class that defines parameters of the output video, including image parameters and compression parameters. Image parameters include the resolution, frame rate, rotation angle, and padding mode, and compression parameters include the encoding format, group of pictures (GOP), bitrate, and video quality. In most cases, you do not need to configure these parameters unless otherwise required.
AliyunMicRecordController A class that defines the microphone controller protocol.
AliyunVideoRecordLayoutParam A class that defines layout parameters of the input source, including the video level and adaptive mode.
AliyunCameraRecordSource A class that defines the input source for camera recording.
AliyunCameraRecordController A class that defines the camera recording controller protocol.
AliyunViewRecordSource A class that defines the input source for screen recording.
AliyunViewRecordController A class that defines the screen recording controller protocol.
AliyunPlayerRecordSource A class that defines parameters of the input source for local videos.
AliyunPlayerRecordController A class that defines the protocol of the local video playback controller.
AliyunClipManager A class that manages video clips, for example, obtains the clip information or deletes video clips.
AliyunRecorderDelegate A class that defines recording delegate callbacks.

Configure multi-source recording

Note The camera and microphone permissions are required for multi-source recording.
Configuration Step Description Sample code
Basic configurations 1 Create a recording instance, configure recording parameters, and add input sources. Initialize the AliyunRecorder class
2 Start preview or stop preview. Configure preview
3 Start recording or stop recording. Start recording
4 Stop recording and generate configuration information. Stop recording
Advanced configurations 5 Configure input source parameters. Manage the input source for camera recording

Manage the input source for screen recording

Manage the input source for local videos

6 Configure background music, background images, speed ramping, and custom rendering. Other features
7 Configure recording callbacks. Configure recording callbacks

Initialize the AliyunRecorder class

Initialize the AliyunRecorder class, create a recording instance, and configure recording parameters. For more information about the parameters that are used in the code, see Related classes.

Configure output parameters

Call AliyunRecorderVideoConfig to configure parameters of the output video, including image parameters and compression parameters. Image parameters include the resolution, frame rate, rotation angle, and padding mode, and compression parameters include the encoding format, GOP, bitrate, and video quality. In most cases, you do not need to configure these parameters unless otherwise required.
AliyunRecorderVideoConfig *videoConfig = [[AliyunRecorderVideoConfig alloc] init];
videoConfig.resolution = CGSizeMake(720, 1280);           // 720P
videoConfig.fps = 30;
...

Add input sources

Create a configuration object by specifying the outputPath parameter in the parameter output object, and then add input sources.
Note
  • You can add one or more input sources based on your business requirements. We recommend that you add no more than three input sources.
  • You can add only one camera recording source.
// Create a configuration object.
AliyunRecorderConfig *config = [[AliyunRecorderConfig alloc] initWithVideoConfig:videoConfig outputPath:[taskPath stringByAppendingPathComponent:@"output.mp4"]];


// Add a camera recording source.
// Specify the camera layout parameters.
AliyunVideoRecordLayoutParam *cameraLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspectFill];
cameraLayout.size = resolution;
cameraLayout.center = CGPointMake(resolution.width / 2.0, resolution.height / 2.0);
cameraLayout.zPosition = 1;
// Add a recording source and return to the controller.
self.cameraRecorderController = [config addCamera:cameraLayout];
self.cameraRecorderController.preview = self.videoView;
self.cameraRecorderController.camera.resolution = AliyunRecordCameraResolution_3840x2160;
self.cameraRecorderController.camera.position = AVCaptureDevicePositionBack;


// Add a screen recording source.
// Specify the layout parameters for screen recording.
AliyunVideoRecordLayoutParam *viewRecordLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspect];
viewRecordLayout.size = CGSizeMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
viewRecordLayout.center = CGPointMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
viewRecordLayout.zPosition = 2;
// Specify the parameters for the screen recording source.
AliyunViewRecordSource *viewSource = [[AliyunViewRecordSource alloc] initWithTargetView:drawView fps:videoConfig.fps];
viewSource.captureInBackground = YES;
// Add a recording source and return to the controller.
self.viewRecordController = [config addViewSource:viewSource layout:viewRecordLayout];


// Add a local playback source.
// Specify the layout parameters for local videos.
AliyunVideoRecordLayoutParam *playerRecordLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspect];
playerRecordLayout.size = CGSizeMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
playerRecordLayout.center = CGPointMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
playerRecordLayout.zPosition = 2;
// Specify the parameters for the local playback source.
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:videoPath] options:nil];
AliyunPlayerRecordSource *playerSource = [[AliyunPlayerRecordSource alloc] initWithAsset:asset fps:videoConfig.fps];
// Add a recording source and return to the controller.
self.playerRecordController = [config addMVSource:playerSource layout:playerRecordLayout];
self.playerRecordController.preview = self.playerView;
Create an instance
// Specify the parameters to create an instance.
AliyunRecorder *recorder = [[AliyunRecorder alloc] initWithConfig:config];
recorder.delegate = self;
recorder.clipManager.maxDuration = 5;
recorder.clipManager.minDuration = 1;
recorder.clipManager.deleteVideoClipsOnExit = YES;
self.aliyunRecorder = recorder;

Configure preview

For more information about the parameters that are used in the code, see Related classes.

// Enable preview.
[self.aliyunRecorder startPreview];

// After the recording is complete, call stopPreview to stop preview.
[self.aliyunRecorder stopPreview];

Start recording

startRecording and stopRecording must be called in pairs. You can call them once or multiple times, and one or more temporary video clips are generated. For more information about the parameters that are used in the code, see Related classes.
// Start recording a video clip.
[self.aliyunRecorder startRecord];  

// Stop recording a video clip. After you perform the operation, recording does not stop immediately. Instead, a change of status occurs. Call onAliyunRecorder:stateDidChange: to obtain the recording status callback.
[self.aliyunRecorder stopRecord];
The following table describes the recording status.
Status Description
AliyunRecorderState_Idle Idle. Recording awaits to be started.
AliyunRecorderState_LoadingForRecord Loading before recording. Callbacks for the first frames of recording sources await to be returned. After the callbacks are all returned, the status changes to Recording.
AliyunRecorderState_Recording Recording in progress.
AliyunRecorderState_Stopping Stopping. Internal buffering awaits to be complete. After internal buffering is complete, the status changes to Stop.
AliyunRecorderState_Stop Recording stopped.
AliyunRecorderState_Error An error occurred. You can cancel recording to reset the status to Idle.

Stop recording

If you stop recording, the recorded video clips are merged into a video or the configuration information about the recorded video clips is generated. For more information about the parameters that are used in the code, see Related classes.
  • finishRecord: merges the recorded video clips into one video. Call this API operation if you do not need to continue editing the generated video.
  • finishRecordForEdit: does not merge the recorded video clips. Call this API operation if you want to continue editing the generated video later. You can initialize the editor by configuring the taskPath parameter. For more information, see Initialize an editor.
// Stop recording and merge the recorded video clips into one video.
[self.aliyunRecorder finishRecord:^(NSString *outputPath, NSError *error) {
    if (!error) {
        // After recording is complete, you can preview, edit, and upload the video in outputPath.
    }
}];

// Stop recording without merging the recorded video clips into one video. taskPath is generated.
[self.aliyunRecorder finishRecordForEdit:^(NSString *taskPath, NSError *error) {
    if (!error) {
        // After recording is complete, you can configure the taskPath parameter to initialize AliyunEditor and edit the video. For more information, see topics about video editing.
    }
}];

Manage the input source for camera recording

After you add a source and return to the controller that is based on the AliyunCameraRecordController protocol, you can configure the parameters, adjust the border, use basic retouching, apply face stickers, static stickers, animated GIFs, static filters, and animated filters, and take photos.

Configuration of common parameters
// Specify whether to turn on the flashlight.
self.cameraRecorderController.camera.torchMode = AVCaptureTorchModeOn;

// Specify the camera position.
self.cameraRecorderController.camera.position = AVCaptureDevicePositionBack;

// Specify the zoom factor.
self.cameraRecorderController.camera.videoZoomFactor = 2.0;

// Specify the exposure level.
self.cameraRecorderController.camera.exposureValue = 0.8;

// Specify the orientation.
self.cameraRecorderController.camera.orientation = UIDeviceOrientationLandscapeLeft;

// Specify the resolution. We recommend that you do not change the value of this parameter. The resolution is automatically adapted based on the output size.
self.cameraRecorderController.camera.resolution = AliyunRecordCameraResolution_1280x720;

// Specify the flash mode. If you set the flash mode to on, the flash is turned on when you take photos.
self.cameraRecorderController.camera.flashMode = AVCaptureFlashModeOn;
Border adjustment
AliyunVideoRecordBorderInfo *cameraBorder = [AliyunVideoRecordBorderInfo new];
cameraBorder.color = UIColor.whiteColor;
cameraBorder.width = 3.0;
cameraBorder.cornerRadius = 10.0;
self.cameraRecorderController.borderInfo = cameraBorder;
Basic retouching
// Enable basic retouching.
self.cameraRecorderController.beautifyStatus = YES;
self.cameraRecorderController.beautifyValue = 80;

// Disable basic retouching.
self.cameraRecorderController.beautifyStatus = NO;
self.cameraRecorderController.beautifyValue = 0;
Face stickers
// Add a face sticker.
[self.cameraRecorderController applyFaceSticker:[self.class resourcePath:@"Gif/hanfumei-800"]];

Filters

You can create custom filters. For more information, see Filters and transitions.

// Add a filter.
AliyunEffectFilter *filter = [[AliyunEffectFilter alloc] initWithFile:path];
[self.cameraRecorderController applyFilter:filter];
Animated filters
// Add an animated filter.
NSString *filterDir = [self.class resourcePath:@"AnimationEffect/split_screen_3"];
AliyunEffectFilter *animationFilter =[[AliyunEffectFilter alloc] initWithFile:filterDir];
[self.cameraRecorderController applyAnimationFilter:animationFilter];
Static stickers
// Add a static sticker.
AliyunImageStickerController *imageController = [self.cameraRecorderController addImageSticker:imagePath];
[imageController beginEdit];
imageController.image.center = CGPointMake(150, 200);
[imageController endEdit];

Animated stickers

You can create custom animated stickers. For more information, see Animations.
// Add an animated sticker.
AliyunGifStickerController * gifController = [self.cameraController addGifStickerWithConfig:[PathTool boundlePathWithPath:@"Resource/Gif/hanfumei-800"]];
[gifController beginEdit];
gifController.gif.center = CGPointMake(150, 200);
[gifController endEdit];
Photo capture
 // Take a photo and asynchronously obtain the following types of images:
 // image: the rendered image that is collected.
 // rawImage: the raw image that is collected.
[self.cameraRecorderController takePhoto:^(UIImage *image, UIImage *rawImage) {
        
}];

Manage the input source for screen recording

After you add a source and return to the controller that is based on the AliyunViewRecordController protocol, you can adjust the border.

Border
AliyunVideoRecordBorderInfo *border = [AliyunVideoRecordBorderInfo new];
border.color = UIColor.whiteColor;
border.width = 3.0;
border.cornerRadius = 10.0;
self.viewRecorderController.borderInfo = border;

Manage the input source for local videos

After you add a source and return to the controller that is based on the AliyunPlayerRecordController protocol, you can adjust the border.

Border
AliyunVideoRecordBorderInfo *border = [AliyunVideoRecordBorderInfo new];
border.color = UIColor.whiteColor;
border.width = 3.0;
border.cornerRadius = 10.0;
self.playerRecordController.borderInfo = border;

Other features

Features such as background music, watermarks, and background settings are supported. For more information about the parameters that are used in the code, see Related classes.

Manage video clips

Call startRecord and stopRecord to record multiple video clips. You can call clipManager to manage the video clips, for example, delete the last video clip or delete all video clips. For more information about operation parameters, see AliyunClipManager.
// Delete the last video clip.
[self.aliyunRecorder.clipManager deletePart];
// Delete all video clips.
[self.aliyunRecorder.clipManager deleteALLPart];

// Obtain the total number of video clips.
[self.aliyunRecorder.clipManager partCount];
Speed ramping
// Specify the recording speed. We recommend that you set this parameter to a value from 0.5 to 2. 
[self.aliyunRecorder setRate:2];
Background music
// Add background music.
AVURLAsset *audioAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:filePath] options:nil];
float audioDuration = CMTimeGetSeconds(audioAsset.duration);
[self.aliyunRecorder.config setBgMusicWithFile:filePath
                                startTime:0.0 // The background music starts from the 0-second position.
                                duration:MIN(self.aliyunRecorder.clipManager.maxDuration, audioDuration)];

// Remove the background music.
[self.aliyunRecorder.config removeBgMusic];
Watermarks
// Generate a watermark.
- (AliyunRecorderImageSticker *) waterMark
{
    if (!_waterMark) {
        NSString *watermarkPath = [AlivcImage pathOfImageName:@"shortVideo_paster_gif"];
        _waterMark = [[AliyunRecorderImageSticker alloc] initWithImagePath:watermarkPath];
        _waterMark.size = CGSizeMake(42, 30);
        _waterMark.center = CGPointMake(_waterMark.size.width * 0.5 + 4, _waterMark.size.height * 0.5 + 4);
        _waterMark.autoresizingMask = UIViewAutoresizingFlexibleRightMargin | UIViewAutoresizingFlexibleBottomMargin;
    }
    return _waterMark;
}

// Apply the watermark.
[self.aliyunRecorder.config addWaterMark:self.waterMark];

// Remove the watermark.
[self.aliyunRecorder.config removeWaterMark:self.waterMark.stickerId];
Background settings
// Generate a background object.
- (AliyunRecorderBackgroundInfo *) bgInfo
{
    if (!_bgInfo) {
        _bgInfo = [AliyunRecorderBackgroundInfo new];
         // _bgInfo.color = UIColor.redColor;
        NSString *imgName = @"xxx.png";
        _bgInfo.image = [UIImage imageNamed:imgName];
        _bgInfo.renderMode = AliyunRenderMode_ResizeAspectFill;
    }
    return _bgInfo;
}

// Apply the background object.
self.aliyunRecorder.config.bgInfo = self.bgInfo;

Custom rendering

After you add a camera recording source, the short video SDK passes the CMSampleBufferRef data that is collected by the camera to the business layer by using customRender. You can import a third-party SDK into the business layer to perform custom rendering of the data and then return the rendering result CVPixelBufferRef to the short video SDK for preview and production.
// Step 1: Configure custom rendering. Note that a camera recording source must be added.
self.aliyunRecorder.customRender = self;
// Step 2: Implement the AliyunRecorderCustomRender protocol.
- (CVPixelBufferRef) onAliyunRecorderCustomRenderToPixelBuffer:(AliyunRecorder *)recorder withSampleBuffer:(CMSampleBufferRef)sampleBuffer {
    // If you integrate Queen SDK to perform custom rendering, sample CVPixelBufferRef is returned.
    // If you do not perform custom rendering, CMSampleBufferGetImageBuffer(sampleBuffer) is returned.
    ...
}
Note Custom rendering allows you to process the data collected by the camera. Related features include face retouching, face shaping, makeup, gesture recognition, AI image matting, and chroma key. Queen SDK provides these features and fits well with the short video SDK. For more information, see Demo of the short video SDK for iOS.

Configure recording callbacks

You can configure callbacks to obtain the processing progress and status of audio and video in a timely manner. For more information about the parameters that are used in the code, see Related classes.

// Configure common callbacks as follows:
#pragma mark - AliyunRecorderDelegate

- (void)onAliyunRecorderWillStopWithMaxDuration:(AliyunRecorder *)recorder {
    NSLog(@"Record2 Will Stop Recording With Max Duration");
    [self.aliyunRecorder stopPreview];
}

- (void)onAliyunRecorderDidStopWithMaxDuration:(AliyunRecorder *)recorder {
    NSLog(@"Record2 Did Recording With Max Duration");
    [self.aliyunRecorder finishRecord:^(NSString *outputPath, NSError *error) {
        if (!error) {

        }
    }];
}

- (void)onAliyunRecorder:(AliyunRecorder *)recorder progressWithDuration:(CGFloat)duration {
    NSLog(@"Record2 Video Duration: %f", duration);
}

- (void)onAliyunRecorder:(AliyunRecorder *)recorder occursError:(NSError *)error {
    NSLog(@"Record2 Occurs Error: %@", error);
}