How can I merge audio from an overlay video with the main video while using fluent-ffmpeg in NodeJS?

How can I modify this code to achieve this audio merging while preserving the video overlay functionality?

I’m utilizing the fluent-ffmpeg library in my NodeJS project to overlay one video onto another, but I also require the audio from the overlay video to be merged with the main video. Here’s a snippet of the code I’m using:

      'filter': 'overlay', 
      'options': {'x':0, 'y':0, 'eof_action': 'pass'},
      'inputs':['0:v', '1:v'],
      'outputs': 'output'
  ], 'output')
  .saveToFile(outputFile) // Output filepath
  .on('end', function (err, stdout, stderr) { // When video generation is complete
  .on('error', function(err) {
    throw new Error(err.message);

This code snippet utilizes fluent-ffmpeg to overlay the pipVideo onto the mainVideo. The complexFilter function is used to apply the overlay filter, specifying the position and behavior. However, it currently only handles video streams, not audio. To also merge the audio from the overlay video with the main video, additional steps are needed.

1 Like

Yes, you can merge audio from an overlay video with the main video using the fluent-ffmpeg library in NodeJS, a sequential process is followed. First, both the main video and the overlay video containing the desired audio are specified as inputs.

Then, the complexFilter method is employed to apply the overlay filter, positioning the overlay video onto the main video. Within this filter, parameters such as overlay coordinates (x and y) and post-file end behavior (eof_action) are defined. The inputs and outputs for the complex filter are then specified, with references to the video streams of the main and overlay videos labeled as ‘0:v’ and ‘1:v’, respectively.

The ‘output’ label is the resulting stream post-overlay application. Following this, the output file path for the merged video is specified using the saveToFile method. Optional event handling for ‘end’ and ‘error’ events is included to signal video generation completion and handle encountered errors. Through the execution of this code, fluent-ffmpeg effectively overlays the video from the overlay input onto the main input while retaining the audio from the overlay video. It is important to ensure accurate input video paths, adjust overlay positioning as necessary, and appropriately manage any potential processing errors.

You can refer to these threads for more understanding:

How can adjustments be made to the volume levels of the overlay video’s audio to ensure harmonious blending with the main video’s audio during the merging process?

To manipulate the volume levels of the overlay video’s audio, FFmpeg offers a range of audio filter options that can be utilized within the complexFilter method. These filters, such as volume, amix, or adelay, enable fine-tuning of audio properties to achieve the desired outcome.

For example, the volume filter allows for precise control over audio volume, ensuring alignment with the main video’s audio level. Similarly, the amix filter facilitates the blending of multiple audio streams into a unified output.

Additionally, the adelay filter aids in synchronizing audio streams to address any timing discrepancies introduced during the overlay process. By strategically applying these audio filters within the complexFilter method, adjustments can be made to the overlay video’s audio, enabling seamless integration with the main video’s audio and enhancing the overall viewing experience.