How can I merge audio from an overlay video with the main video while using fluent-ffmpeg in NodeJS?

[michael]

How can I modify this code to achieve this audio merging while preserving the video overlay functionality?

I’m utilizing the fluent-ffmpeg library in my NodeJS project to overlay one video onto another, but I also require the audio from the overlay video to be merged with the main video. Here’s a snippet of the code I’m using:

ffmpeg()
  .input(mainVideo)
  .input(pipVideo)
  .complexFilter([
    {
      'filter': 'overlay', 
      'options': {'x':0, 'y':0, 'eof_action': 'pass'},
      'inputs':['0:v', '1:v'],
      'outputs': 'output'
    }
  ], 'output')
  .saveToFile(outputFile) // Output filepath
  .on('end', function (err, stdout, stderr) { // When video generation is complete
    console.log('done');
  })
  .on('error', function(err) {
    throw new Error(err.message);
  });

This code snippet utilizes fluent-ffmpeg to overlay the pipVideo onto the mainVideo. The complexFilter function is used to apply the overlay filter, specifying the position and behavior. However, it currently only handles video streams, not audio. To also merge the audio from the overlay video with the main video, additional steps are needed.

[preetpal]

Yes, you can merge audio from an overlay video with the main video using the fluent-ffmpeg library in NodeJS, a sequential process is followed. First, both the main video and the overlay video containing the desired audio are specified as inputs.

Then, the complexFilter method is employed to apply the overlay filter, positioning the overlay video onto the main video. Within this filter, parameters such as overlay coordinates (x and y) and post-file end behavior (eof_action) are defined. The inputs and outputs for the complex filter are then specified, with references to the video streams of the main and overlay videos labeled as ‘0:v’ and ‘1:v’, respectively.

The ‘output’ label is the resulting stream post-overlay application. Following this, the output file path for the merged video is specified using the saveToFile method. Optional event handling for ‘end’ and ‘error’ events is included to signal video generation completion and handle encountered errors. Through the execution of this code, fluent-ffmpeg effectively overlays the video from the overlay input onto the main input while retaining the audio from the overlay video. It is important to ensure accurate input video paths, adjust overlay positioning as necessary, and appropriately manage any potential processing errors.

You can refer to these threads for more understanding:

Why is fluent-ffmpeg not concatenating my files?

I’m attempting to create a single MP4 file out of many MP3 and JPG files. But when I run the code, it either creates an empty MP4 file or one that only has the audio from the original MP3 file. Despite numerous code modifications, the desired result is still elusive. Is there anyone who can pinpoint the problem with my strategy? Here is the code, const path = require("path"); const ffmpeg = require("fluent-ffmpeg"); let newMp4 = ffmpeg(); newMp4 .input(`${dirPath}/darkpatio.jpg`) …

How to merge multiple videos using fluent-ffmpeg?

We have built our video transcoding pipeline in NodeJS using the fluent-ffmpeg node module, what kind of parameters do I have to pass for merging 2 or more videos into a single mp4 file with fluent-ffmpeg?

How can I encode/convert multiple video files one by one using fluent ffmpeg?

I have a nodejs script which encodes/converts multiple video files using the fluent-ffmpeg package. Since FFmpeg is very resource intensive my Linux server runs out of resources and my scripts start throwing an error. This is because all the videos are encoded concurrently. Is there a way to encode the videos one at a time and not concurrently? My code snippet looks like this: const Ffmpeg = require(“fluent-ffmpeg”); videos.forEach(video=> { covertVideo(filename, (compressionResult)=>{ //d…

[michael]

How can adjustments be made to the volume levels of the overlay video’s audio to ensure harmonious blending with the main video’s audio during the merging process?

[preetpal]

To manipulate the volume levels of the overlay video’s audio, FFmpeg offers a range of audio filter options that can be utilized within the complexFilter method. These filters, such as volume, amix, or adelay, enable fine-tuning of audio properties to achieve the desired outcome.

For example, the volume filter allows for precise control over audio volume, ensuring alignment with the main video’s audio level. Similarly, the amix filter facilitates the blending of multiple audio streams into a unified output.

Additionally, the adelay filter aids in synchronizing audio streams to address any timing discrepancies introduced during the overlay process. By strategically applying these audio filters within the complexFilter method, adjustments can be made to the overlay video’s audio, enabling seamless integration with the main video’s audio and enhancing the overall viewing experience.