How do we resolve issues related to applying various filters while transcoding video with hardware acceleration?

I have been trying to transcode a video applying some video filter in FFmpeg with NVIDIA hardware acceleration and receiving the following error from FFmpeg.
ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i in.mp4 -vf v360=input=sg:ih_fov=118.2:iv_fov=69.5:output=hequirect:w=2924:h=2924 -c:v h264_nvenc -preset slow -b:v 102400k -bufsize 5000k -c:a copy out.mp4

Impossible to convert between the formats supported by the filter ‘graph 0 input from stream 0:0’ and the filter ‘auto_scale_0’
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0

What does this error message mean and how to resolve it?

1 Like

To apply the various filters while transcoding video with hardware in FFmpeg, you need to check where a particular video filter performs its operations, either CPU or the device used for hardware acceleration.

If your video filter does computations on the CPU then your video frames should be main memory (RAM) and if your video filter does a computation on the device used for hardware acceleration then video frames should reside in the device’s memory. It is necessary for faster computation otherwise it can add memory transfer delay.

FFmpeg provides hwupload and hwdownload filters for moving video frames from RAM and hardware-accelerated devices and vice versa. hwdownload filter allows you to move video frames from the device used for hardware acceleration to RAM and hwupload allows you to upload video frames from RAM to a particular device.

Also, frame format matters here. When you decode your video using hardware acceleration you will have video frames in the device used for hardware acceleration in the format supported by the hardware decoder. If your video filter does not support the format used by the hardware decoder then you have to change the video frame format using format filter.

You have used the -hwaccel_output_format cuda option in your command which tells the FFmpeg decoder to output frames in cuda since cuda format is not supported by your video filter it gives you the error.

You can resolve this error by either removing -hwaccel_output_format cuda from your command or adding hwdownload and format filters before your video filters like in the following example.

ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i in.mp4 -vf hwdownload,format=nv12,v360=input=sg:ih_fov=118.2:iv_fov=69.5:output=hequirect:w=2924:h=2924 -c:v h264_nvenc -preset slow -b:v 102400k -bufsize 5000k -c:a copy out.mp4

Why, despite Chrome’s hardware acceleration setting being off, is it still possible for Discord users to stream media from websites like Netflix and TV shows?

Because the HDCP content can be kept encrypted all the way to the display thanks to the hardware acceleration, when it is disabled, the video is often decrypted in software at a lower resolution and/or frame rate.