Create a video file from an audio file and add visualizations from audio

Question

My initial thought was to upload audio files to YouTube along with video that is inspired from the audio. The particular visualization can be in different form such as spectrum, spectogram, or other forms of visualizations that change with the audio. I'm not familiar with all the capabilities of ffmpeg or sox, but I wonder if I can do something like this out of the box, or as a series of scripts with other command line utilities.

Solution 1

Here are some examples for taking an audio file, running it through ffmpeg, and have a video created based on some of the filters available in ffmpeg.

Examples:

spectogram:

ffmpeg -i song.mp3 -filter_complex showspectrum=mode=separate:color=intensity:slide=1:scale=cbrt -y -acodec copy video.mp4

avectorscope:

ffmpeg -i song.mp3 -filter_complex avectorscope=s=320x240 -y -acodec copy video.mp4

zooming mandelbrot:

ffmpeg -i song.mp3 -f lavfi -i mandelbrot=s=320x240 -y -acodec copy video.mp4

(Screenshot missing)

source: [Libav-user] ffmpeg showspectrum to file

Solution 2

Audio visualization with ffmpeg

ffmpeg -i input.mp3 -filter_complex \
"[0:a]avectorscope=s=640x518,pad=1280:720[vs]; \
 [0:a]showspectrum=mode=separate:color=intensity:scale=cbrt:s=640x518[ss]; \
 [0:a]showwaves=s=1280x202:mode=line[sw]; \
 [vs][ss]overlay=w[bg]; \
 [bg][sw]overlay=0:H-h,drawtext=fontfile=/usr/share/fonts/TTF/Vera.ttf:fontcolor=white:x=10:y=10:text='\"Song Title\" by Artist'[out]" \
-map "[out]" -map 0:a -c:v libx264 -preset fast -crf 18 -c:a copy output.mkv

ffmpeg can use several filters to visualize audio: avectorscope, showspectrum, and showwaves. You can then place them where you want with overlay, and then add text with drawtext.

In the example above the audio is stream copied (re-muxed) instead of being re-encoded.

From FFmpeg Wiki: How to Encode Videos for YouTube and other Video Sharing Sites.

Comments

[Sun]

+1 for the link so I could search on ffmpeg showspectrum - the FFmpeg examples are too complicated for me.

[Elisa Cha Cha]

@sunk818 It just takes some practice. You can just copy and paste the command and it will do as shown above. You may have to adjust the fontfile if you decide you want to add text too, or just remove the drawtext part.

[Sun]

the fontfile gave me an error and I wasn't too interested in figuring out the syntax for Windows

[Ryan]

I get Codec not supported: VLC could not decode the format " " (No description for this codec) unless I change "mp4" to "mkv". But +1 anyway because these were helpful examples.

[Flimm]

Another one is showwaves: ffmpeg -i input.mp3 -filter_complex showwaves=s=1280x202:mode=line -acodec copy video.mp4