Video editing preparations - Intelligent Media Services - Alibaba Cloud Documentation Center

Learn how to use the Intelligent Media Services (IMS) SDK to perform batch video production by calling the SubmitMediaProducingJob API.

What is batch video production

Batch video production allows you to create multiple unique videos by combining a group of audio and video assets with voiceovers.

Use case

As more businesses use short video platforms for marketing, they need to distribute ads across multiple key opinion leader (KOL) and marketing accounts. To avoid deduplication mechanisms on these platforms that block identical content, each video must be unique. Batch video production solves this by allowing you to create many variations from a set of media assets. You can achieve different production results by customizing the Timeline parameter in the SubmitMediaProducingJob API.

Billing

Submitting a video editing and production job may incur charges for multiple billable items. For details, see Intelligent production.

How it works

Combine one or more video, audio, image, and subtitle assets into a final video by submitting editing jobs with different timeline parameter configurations using the SubmitMediaProducingJob API.
A timeline organizes your media assets and effects according to your creative vision. It primarily consists of three object types: tracks, clips, and effects. For more information, see Timeline configurations.
Intelligent production supports editing, effect rendering, and template-based production for live streams, on-demand videos, and asset files in OSS. For more information, see Intelligent production overview.

Before you begin

Activate IMS.
Create a RAM user and obtain its AccessKey pair. For more information, see Create and grant permissions to a RAM user.

Step 1: Create an OSS bucket

Log on to the OSS Console, choose Buckets > Create Bucket, and set Region to China (Shanghai). Take note of the bucket name, as it will be used as the output destination for the final videos and is required in the following steps.

Step 2: Run the code

Environment requirements:
- Operating System (OS): Windows 8.1 or later, macOS, or Linux.
- IntelliJ IDEA 2020.1 or later.
- JDK 1.8 or later.
Create a new Maven project to run the code.

Import the required third-party dependencies.

Note

The SDK version in the following sample code is for reference only. For the latest version, see Server-side SDK.

<dependencies>
  <dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>ice20201109</artifactId>
    <version>2.3.0</version>
  </dependency>
  <dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.9</version>
  </dependency>
</dependencies>

Configure your AccessKey ID and AccessKey Secret. For instructions, see Manage access credentials.
Note
For more ways to manage your credentials, see Manage access credentials.

Create a new Java class and paste the following sample code.

Expand to view sample code

import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyun.ice20201109.Client;
import com.aliyun.ice20201109.models.*;
import com.aliyun.teaopenapi.models.Config;

import java.util.*;

// Video URL
// http://oushu-test-shanghai.oss-cn-shanghai.aliyuncs.com/ice_output/46b29eb5775f4f758846171ab79bfca7.mp4

/**
 *  Maven dependencies required:
 *   <dependency>
 *      <groupId>com.aliyun</groupId>
 *      <artifactId>ice20201109</artifactId>
 *      <version>2.3.0</version>
 *  </dependency>
 *  <dependency>
 *      <groupId>com.alibaba</groupId>
 *      <artifactId>fastjson</artifactId>
 *      <version>1.2.9</version>
 *  </dependency>
 */
public class BatchProduceAlignment {

    static final String regionId = "cn-shanghai";
    static final String bucket = "The bucket you created in Step 1";
    private Client iceClient;
    
    public static void main(String[] args) {
        try {
            BatchProduceAlignment batchProduce = new BatchProduceAlignment();
            batchProduce.initClient();
            batchProduce.run();
        } catch (Exception e) {
            System.out.println("Produce failed. Exception: " + e.toString());
        }
    }

    public void initClient() throws Exception {

        // An AccessKey pair of an Alibaba Cloud account has permissions to call all API operations. We recommend that you use a RAM User to call API operations or perform routine O&M. 
        // This example shows how to store your AccessKey ID and AccessKey Secret in environment variables. For more information about how to configure environment variables, see https://www.alibabacloud.com/help/en/sdk/developer-reference/v2-manage-access-credentials
        com.aliyun.credentials.Client credentialClient = new com.aliyun.credentials.Client();
        Config config = new Config();
        config.setCredential(credentialClient);

        // If you want to hardcode the AccessKey ID and AccessKey Secret, use the following code. However, we strongly recommend that you do not save them in your project code to avoid security risks.
        // config.accessKeyId = <your AccessKey ID>;
        // config.accessKeySecret = <your AccessKey Secret>;
        config.endpoint = "ice." + regionId + ".aliyuncs.com";
        config.regionId = regionId;
        iceClient = new Client(config);
    }

    public void run() throws Exception {
        // Text material.
        String text = "People describe life with mixed flavors because they understand that taste is a sentiment cherished in everyone's heart. In this era, everyone has experienced too much pain and joy. People tend to hide the bitterness in their hearts and turn happiness into food, presenting it on the dining table throughout the seasons.";
        // Video materials.
        String[] videoArray = new String[]{
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f1.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f2.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f3.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f4.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f5.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f6.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f7.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f8.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f9.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f10.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f11.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f12.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f13.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f14.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f15.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f16.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f17.mp4",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/food/f18.mp4"
        };
        // Background music materials.
        String[] bgMusicArray = new String[]{
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m2.wav",
                "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m3.wav"
        };
        // Video title.
        String title = "Delicacies on the tip of the tongue";
        // Number of videos to produce.
        int produceCount = 3;

        // Submit jobs in a batch.
        List<String> jobIds = new ArrayList<String>();
        for (int i = 0; i < produceCount; i++) {
            String jobId = produceSingleVideo(title, text, videoArray, bgMusicArray);
            jobIds.add(jobId);
        }
        // Poll job status until all jobs are complete.
        System.out.println("waiting for jobs to finish...");
        while (true) {
            Thread.sleep(3000);
            boolean allFinished = true;
            for (int i = 0; i < jobIds.size(); i++) {
                String jobId = jobIds.get(i);
                GetMediaProducingJobRequest req = new GetMediaProducingJobRequest();
                req.setJobId(jobId);
                GetMediaProducingJobResponse response = iceClient.getMediaProducingJob(req);
                GetMediaProducingJobResponseBody.GetMediaProducingJobResponseBodyMediaProducingJob mediaProducingJob = response.getBody().getMediaProducingJob();
                String status = mediaProducingJob.getStatus();
                System.out.println("jobId: " + mediaProducingJob.getJobId() + ", status: " + status);
                if ("Failed".equalsIgnoreCase(status)) {
                    System.out.println("Job failed. jobInfo: " + JSONObject.toJSONString(mediaProducingJob));
                    throw new Exception("Production failed. jobid: " + mediaProducingJob.getJobId());
                }
                if (!"Success".equalsIgnoreCase(status)) {
                    allFinished = false;
                    break;
                }
            }
            if (allFinished) {
                System.out.println("All jobs finished.");
                break;
            }
        }
    }

    public String produceSingleVideo(String title, String text, String[] videoArray, String[] bgMusicArray) throws Exception {
        text = text.replace("，", "。");
        text = text.replace("\n", "。");
        String[] sentenceArray = text.split("。");

        JSONArray videoClipArray = new JSONArray();
        JSONArray audioClipArray = new JSONArray();

        List<String> videoList = Arrays.asList(videoArray);
        Collections.shuffle(videoList);

        for (int i = 0; i < sentenceArray.length; i++) {
            String sentence = sentenceArray[i];
            String clipId = "clip" + i;
            String videoUrl = videoList.get(i);
            String videoClip = "{\"MediaURL\":\""+videoUrl+"\",\"ReferenceClipId\":\""+clipId+"\",\"Effects\":[{\"Type\":\"Background\",\"SubType\":\"Blur\",\"Radius\":0.1}]}";
            videoClipArray.add(JSONObject.parseObject(videoClip));
            String audioClip = "{\"Type\":\"AI_TTS\",\"Content\":\"" + sentence + "\",\"Voice\":\"zhichu\",\"ClipId\":\""+clipId+"\",\"Effects\":[{\"Type\":\"AI_ASR\",\"Font\":\"Alibaba PuHuiTi\",\"Alignment\":\"TopCenter\",\"Y\":0.75,\"FontSize\":55,\"FontColor\":\"#ffffff\",\"AdaptMode\":\"AutoWrap\",\"TextWidth\":0.8,\"Outline\":2,\"OutlineColour\":\"#000000\"}]}";
            audioClipArray.add(JSONObject.parseObject(audioClip));
        }

        String subtitleTrack = "{\"SubtitleTrackClips\":[{\"Type\":\"Text\",\"Font\":\"HappyZcool-2016\",\"Content\":\""+title+"\",\"FontSize\":80,\"FontColor\":\"#ffffff\",\"Y\":0.15,\"Alignment\":\"TopCenter\",\"EffectColorStyle\":\"CS0004-000005\",\"FontFace\":{\"Bold\":true,\"Italic\":false,\"Underline\":false}}]}";

        int bgMusicIndex = (int)(Math.random() * bgMusicArray.length);
        String bgMusicUrl = bgMusicArray[bgMusicIndex];
        String timeline = "{\"VideoTracks\":[{\"VideoTrackClips\":"+videoClipArray.toJSONString()+"}],\"AudioTracks\":[{\"AudioTrackClips\":"+audioClipArray.toJSONString()+"},{\"AudioTrackClips\":[{\"MediaURL\":\""+bgMusicUrl+"\"}]}],\"SubtitleTracks\":[" + subtitleTrack + "]}";

        //
        String targetFileName = UUID.randomUUID().toString().replace("-", "");
        String outputMediaUrl = "http://" + bucket + ".oss-" + regionId + ".aliyuncs.com/ice_output/" + targetFileName + ".mp4";
        int width = 720;
        int height = 1280;
        String outputMediaConfig = "{\"MediaURL\":\"" + outputMediaUrl + "\",\"Width\":"+width+",\"Height\":"+height+"}";

        SubmitMediaProducingJobRequest request = new SubmitMediaProducingJobRequest();
        request.setTimeline(timeline);
        request.setOutputMediaConfig(outputMediaConfig);
        SubmitMediaProducingJobResponse response = iceClient.submitMediaProducingJob(request);
        System.out.println("Started job. jobid: " + response.getBody().getJobId() + ", outputMediaUrl: " + outputMediaUrl);
        return response.getBody().getJobId();
    }

}

Replace the bucket placeholder in the code with the name of the bucket you created in Step 1.
Click Run to generate the videos. The overall workflow of the code is as follows: Generate a timeline -> Submit a video production job -> Poll the job status -> Print logs to the console.
outputMediaUrl is the output address. When the job status is Success, you can preview the final video. Since the bucket is private, the URL is not directly accessible. To preview the video, go to the OSS console, find the output file, and click Copy Object URL to get a signed URL.
The produceCount variable in the code specifies the number of videos to produce. You can replace the sample text and video assets in the code with your own content. For more information, see Timeline configurations.

References

Next steps

To edit video and image materials, refer to Image and video editing.
To process audio by modifying the timeline, refer to Audio processing.
To add subtitles and use subtitle templates, refer to Subtitles and subtitle templates.
To add transitions, effects, and filters, refer to Transitions, effects, and filters.
To incorporate intelligent processing into your video production, refer to Intelligent data processing.
For other settings you can configure, refer to Other settings.