Promo Center

50% off for new user

Direct Mail-46% off

Learn More

Audio processing

Updated at: 2024-12-09 08:01

This topic describes how to configure the Timeline parameter when you call the SubmitMediaProducingJob operation in different audio processing scenarios.

Usage notes

  • Intelligent production supports editing and compositing, effect rendering, and templates for live streams, VOD files, and material files from Object Storage Service (OSS). For more information, see Intelligent production overview.

  • You can produce a video from one or more videos, audio files, images, and subtitle materials by configuring Timeline parameters and calling the SubmitMediaProducingJob operation.

  • A timeline is created when you add materials and configure effects to create a video. A timeline consists of tracks, materials, and effects. For more information, see Timeline configurations.

  • For more information about how to use the IMS SDK to edit audio and video files, see Preparations.

Audio processing for videos

Mute an entire video

You can configure the Effects element to mute an entire video.

Note

The Gain field specifies a gain level. The value 0 specifies that the video is muted.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0
            }
          ]
        }
      ]
    }
  ]
}

Adjust the volume

You can adjust the volume for audio and video clips.

Note
  • The following sample code sets the gain level of the video clip to 0.5 and the gain level of the audio clip to 2.

  • The Gain field specifies a gain level. You can set the gain level to one of the following values:

    • 0: mute.

    • 1: original volume.

    • A value between 0 and 1: less than the original volume but not muted. A smaller value indicates a smaller gain level.

    • A value greater than 1: greater than the original volume. A larger value indicates a greater gain level.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0.5
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 2
            }
          ]
        }
      ]
    }
  ]
}

Mute an entire video and mix audio

A common audio processing scenario is to mute an entire video and then mix audio into the video.

Note
  • All audio clips to use must be placed within the AudioTracks element.

  • If the TimelineIn field is not specified, the audio mixing starts from the beginning of the video clip.

  • If the TimelineOut field is not specified, the entire audio clip is used for mixing. If the audio clip is longer than the video clip, the audio section to mix into the video has the same length as the video clip and the trailing audio section is truncated.

  • The AudioTrackClips element can contain audio-only clips or video clips that have audio.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h1.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav"
        }
      ]
    }
  ]
}

Mute an entire video and mix audio from multiple audio clips

You can mute an entire video and mix audio from multiple audio clips.

Note
  • The following sample code mutes the entire video and mixes audio from two audio clips:

    • The audio section from the 1st second to the 5th second in the m1.wav clip is mixed into the section from the 2nd second to the 6th second in the video clip, with the audio gain level set to 2.

    • The audio section from the 2nd second to the 10th second in the speech.mp3 clip is mixed into the section from the 4th second to the 12th second in the video clip, with the audio gain level set to 0.5.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h1.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "In": 1,
          "Out": 5,
          "TimelineIn": 2,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 2
            }
          ]
        }
      ]
    },
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/speech.mp3",
          "In": 2,
          "Out": 10,
          "TimelineIn": 4,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0.5
            }
          ]
        }
      ]
    }
  ]
}

Configure audio loop playback

Note

The following sample code mutes the video clip and repeatedly plays the section of the m2.wav audio from the 4th second to the 10th second over the timeline part of the 2nd second to the 14th second.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m2.wav",
          "LoopMode": true,
          "In": 4,
          "Out": 10,
          "TimelineIn": 2,
          "TimelineOut": 14
        }
      ]
    }
  ]
}

Audio fade-in and fade-out

Configure fade-in and fade-out for an audio-only clip

You can set the Type field to AFade to configure audio fade-in and fade-out.

Note
  • The following sample code sets a fade-in effect for 3 seconds and a fade-out effect for 4 seconds. Fade-in increases the volume linearly, whereas fade-out decreases the volume linearly.

  • For more information about the AFade effect, see AFade.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "Effects": [
            {
              "Type": "AFade",
              "SubType": "In",
              "Duration": 3,
              "Curve": "tri"
            },
            {
              "Type": "AFade",
              "SubType": "Out",
              "Duration": 4,
              "Curve": "tri"
            }
          ]
        }
      ]
    }
  ]
}

Configure audio fade-in and fade-out for a video clip

You can apply fade-in and fade-out effects to audio in a video. The configurations are the same as those of fade-in and fade-out for an audio-only clip.

Note

The following sample code sets a fade-in effect for 3 seconds of the audio track and a fade-out effect for 4 seconds of the audio track. Fade-in increases the volume linearly, whereas fade-out decreases the volume linearly.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "AFade",
              "SubType": "In",
              "Curve": "tri",
              "Duration": 3
            },
            {
              "Type": "AFade",
              "SubType": "Out",
              "Duration": 4,
              "Curve": "tri"
            }
          ]
        }
      ]
    }
  ]
}

Mute a specific section of an audio track

You can mute a specific section of an audio track by specifying the start position and end position to mute.

Note

The following sample code mutes the audio section from the 2nd second to the 5th second.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0,
              "In": 2,
              "Out": 5
            }
          ]
        }
      ]
    }
  ]
}

Specify the start position of audio mix

When you mute an video and mix audio into the video, you can specify the start position and end position of the mix.

Note

The following sample code mixes the audio section of the 10th second to the 19th second into the video starting from the 5th second of the video.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "In":10,
          "Out":19,
          "TimelineIn":5
        }
      ]
    }
  ]
}

Reduce noise

You can reduce noise in audio-only and video clips.

Note

For more information about values of the Mode field, see ADenoise.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "ADenoise",
              "Mode": 1
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "Effects": [
            {
              "Type": "ADenoise",
              "Mode": 1
            }
          ]
        }
      ]
    }
  ]
}

Mix audio in a video

You can mix multiple audio tracks into a video.

Note
  • In the following example, the gain level of the sections from the 4th second to the 6th second and from the 9th second to the 14th second in the h4.mp4 clip is set to 0.8.

  • In addition, audio from two audio tracks is mixed into the video.

    • The audio section from the 10th second to the 17th second in the m1.wav clip is mixed into the section from the 4th second to the 11th second in the video clip, with the audio gain level set to 2.

    • The audio section from the 15th second to the 20th second in the speech.mp3 clip is mixed into the section from the 9th second to the 14th second in the video clip.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "VideoTracks": [
    {
      "VideoTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 0,
              "In": 4,
              "Out": 6
            },
            {
              "Type": "Volume",
              "Gain": 0.8,
              "In": 9,
              "Out": 14
            }
          ]
        }
      ]
    }
  ],
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "In": 10,
          "Out": 17,
          "TimelineIn": 4,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 2
            }
          ]
        }
      ]
    },
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/speech.mp3",
          "In": 15,
          "Out": 20,
          "TimelineIn": 9,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 1
            }
          ]
        }
      ]
    }
  ]
}

Normalize audio

Note

This setting applies only to audio and video clips. If ALoudNorm is used together with ADenosie or Volume, the settings take effect in the following order: ADenoise > Volume > ALoudNorm.

  • You can normalize the volume of the audio.

    Effect

    Timeline example

    Effect

    Timeline example

    Audio processing

    {
      "AudioTracks": [
        {
          "AudioTrackClips": [
            {
              "Comment": "Normalize the volume of the audio",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
              "Effects": [
                {
                  "Type": "ALoudNorm",
                  "Loudness": -24.0,
                  "TruePeak": -2.0
                }
              ]
            },
            {
              "Comment": "Normalize the volume of the audio",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m2.wav",
              "Effects": [
                {
                  "Type": "ALoudNorm",
                  "Loudness": -24.0,
                  "TruePeak": -2.0
                }
              ]
            }
          ]
        }
      ]
    }
  • You can normalize the volume of audio in a video.

    Effect

    Timeline example

    Effect

    Timeline example

    Audio processing

    {
      "VideoTracks": [
        {
          "VideoTrackClips": [
            {
              "Comment": "Normalize the volume of audio in a video",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4",
              "Effects": [
                {
                  "Type": "ALoudNorm",
                  "Loudness": -24.0,
                  "TruePeak": -2.0
                }
              ]
            },
            {
              "Comment": "Normalize the volume of audio in a video",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h1.mp4",
              "Effects": [
                {
                  "Type": "ALoudNorm",
                  "Loudness": -24.0,
                  "TruePeak": -2.0
                }
              ]
            }
          ]
        }
      ]
    }

Equalize the volume

Note

This setting takes effect globally and cannot take effect for a single clip.

  • You can equalize the volume of audio in a video.

    Effect

    Timeline example

    Effect

    Timeline example

    Audio processing

    {
      "VideoTracks": [
        {
          "VideoTrackClips": [
            {
              "Comment": "Equalize the volume of audio in a video",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4"
            },
            {
              "Comment": "Equalize the volume of audio in a video",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h1.mp4"
            }
          ]
        }
      ],
      "EffectTracks": [
        {
    			"EffectTrackItems": [
            {
              "Type": "AEqualize",
              "Peak": 0.95,
              "Gain": 10
            }
          ]
        }
      ]
    }
  • You can equalize the volume of audio.

    Effect

    Timeline example

    Effect

    Timeline example

    Audio processing

    {
      "AudioTracks": [
        {
          "AudioTrackClips": [
            {
              "Comment": "Equalize the volume of audio",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav"
            },
            {
              "Comment": "Equalize the volume of audio",
              "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m2.wav"
            }
          ]
        }
      ],
      "EffectTracks": [
        {
    			"EffectTrackItems": [
            {
              "Type": "AEqualize",
              "Peak": 0.95,
              "Gain": 10
            }
          ]
        }
      ]
    }

Audio-only output clips

Extract audio

You can extract audio from a video clip by specifying the video clip in the AudioTrackClips field of the AudioTracks element.

Effect

Timeline example

Effect

Timeline example

{
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/h4.mp4"
        }
      ]
    }
  ]
}

Merge audio clips

You can merge specified sections of two audio clips in a particular order into one audio track.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "In": 0,
          "Out": 12
        },
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/speech.mp3",
          "In": 12,
          "Out": 20
        }
      ]
    }
  ]
}

Mix audio from multiple audio-only clips

You can mix audio from multiple audio-only clips.

Effect

Timeline example

Effect

Timeline example

Audio processing

{
  "AudioTracks": [
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
          "In": 10,
          "Out": 20,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 2
            }
          ]
        }
      ]
    },
    {
      "AudioTrackClips": [
        {
          "MediaURL": "https://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/speech.mp3",
          "In": 5,
          "Out": 15,
          "Effects": [
            {
              "Type": "Volume",
              "Gain": 1
            }
          ]
        }
      ]
    }
  ]
}

References

  • On this page (1, T)
  • Usage notes
  • Audio processing for videos
  • Mute an entire video
  • Adjust the volume
  • Mute an entire video and mix audio
  • Mute an entire video and mix audio from multiple audio clips
  • Configure audio loop playback
  • Audio fade-in and fade-out
  • Mute a specific section of an audio track
  • Specify the start position of audio mix
  • Reduce noise
  • Mix audio in a video
  • Normalize audio
  • Equalize the volume
  • Audio-only output clips
  • Extract audio
  • Merge audio clips
  • Mix audio from multiple audio-only clips
  • References
Feedback
phone Contact Us

Chat now with Alibaba Cloud Customer Service to assist you in finding the right products and services to meet your needs.

alicare alicarealicarealicare