How can I validate a video file from a script?

Question

I have a server with lots of video files. After a restore, I noticed that the checksum of a couple of files changed. Since I don't have checksums for all files, I wanted write a script to verify the file integrity. It's simple for archives (tar t, unzip -t, rar t, etc) or images (convert image.jpg /tmp/test.png).

Which options do I need to pass to mplayer or vlc or any other video tool on Linux to achieve the same effect (i.e. validate the file contents without having to watch the whole video)?

Solution 1

It sounds like what you want to do is:

mplayer -vo null -ao null input.file

and then parse the output and return value to see if it could actually play & decode the stream. This will take some time (but be faster than realtime). If you want something even faster, here are some more suggestions:

One easy thing is going to be to do an

mplayer -identify -vo null -ao null

on the file, and then parse the output and look at the return value for something that looks reasonable.

With respect to the checksums being incorrect, it's going to be hard to know if this is an issue for your media player or not (mplayer, vlc, totem, etc.). A good media player will tolerate many bit or byte level errors with little impact on the resulting playback. A very strict media player will exit when it sees malformed or incorrect codec & wrapper bytes.

To verify the wrapper (container) bytes, you could do something like

mencoder -ovc copy -oac copy input.file -o output.file

The problem is that mencoder will want to create an .avi file for output. If your inputs are .avi, then this will work great.

You can run a similar ffmpeg commandline, like this:

ffmpeg -acodec copy -vcodec copy input.file output.file

If the files are .mp4 files, you might want to take a look at mp4box ( http://www.videohelp.com/tools/mp4box ) for doing a similar task. The matroska tools are also good for this kind of thing. ( http://www.matroska.org/ )

Solution 2

If you are working with MP4 files you may want to have a look at the mpeg4ip project, specifically the tools like mp4videoinfo or mp4info. This may be enough to meet your needs, and is very quick.

From the front page:

• mp4dump Utility to dump MP4 file meta-information in text form
• mp4trackdump Utility to dump MP4 file track information in text form
• mp4info Utility to display MP4 file summary
• mp4videoinfo Utility to dump information about MP4 file video tracks
• avidump Utility to display AVI file summary
• yuvdump Utility to display a raw video file on the screen
• mpeg_ps_info Utility to display streams in an mpeg program stream or vob file
• mpeg_ps_extract Utility to extract elementary streams in an mpeg program stream or vob file

Here is some sample output of a MP4 taken on my Nokia N95:

manoa:Movies stu$ mp4info 20081017001.mp4 
mp4info version 1.5.0.1
20081017001.mp4:
Track   Type    Info
1   video   MPEG-4 Unknown Profile(4), 3.620 secs, 2700 kbps, 640x480 @ 23.480663 fps
2   audio   MPEG-4 AAC LC, 3.797 secs, 97 kbps, 48000 Hz
manoa:Movies stu$ 
manoa:Movies stu$ 
manoa:Movies stu$ mp4videoinfo 20081017001.mp4
mp4videoinfo version 1.5.0.1
tracks 1
mp4file 20081017001.mp4, track 1, samples 85, timescale 30000
sampleId      1, size 24110 time 0(0) VOP-I
sampleId      2, size  9306 time 4076(135) VOP-P
sampleId      3, size 13071 time 5104(170) VOP-P
... (a bunch more frames and a bit of info)  ...
sampleId     59, size  8702 time 64975(2165) VOP-P
sampleId     60, size  8826 time 65980(2199) VOP-P
sampleId     61, size  9819 time 66966(2232) GOV VOP-I
sampleId     62, size  5591 time 67986(2266) VOP-P
... (a bunch more frames and a bit of info)  ...
sampleId     83, size 10188 time 105546(3518) VOP-P
sampleId     84, size  6533 time 106585(3552) VOP-P
sampleId     85, size  6032 time 107601(3586) VOP-P
manoa:Movies stu$

Solution 3

Short of watching all the videos, there's no "perfect" way to do this.

Video files are quite robust - as an experiment I took a random MPEG-4 video file, opened it in a hex-editor and started changing bytes.. mplayer and Quicktime still played it back without errors.

I had to delete thousands of bytes before getting any error from mplayer:

...
[mpeg4 @ 0x6762b0]marker does not match f_code
[mpeg4 @ 0x6762b0]marker does not match f_code
[mpeg4 @ 0x6762b0]concealing 852 DC, 852 AC, 852 MV errors
[mpeg4 @ 0x6762b0]header damaged:  0.055  16/ 16 15%  1%  3.5% 0 0 
Error while decoding frame!

It wouldn't be difficult to write a script that runs mplayer on each video, and checks the output for error messages/warnings, but unless the changed bytes are in the file header, or a lot of data was changed, you'll never find them all

Solution 4

As mplayer has options to convert from one video format to another that might be good enough for such a test assuming mencoder returns an error if it can not decode the input file (I have not tested that). This would work similar to the image test you mentioned (convert image.jpg /tmp/test.png)

View more solutions

Comments

[johnny]

Is the video corrupt (won't play) if it doesn't have the same checksum? Just curious why you need it so I can learn.

[slacy]

How do you suspect that the files got corrupted? Finding the "Root cause" of such issues might be time better spent. See my answer below for some more suggestions on validating the file.

[Aaron Digulla]

@johnny: I backup my server on harddisks. It seems one of the backup harddisks had a fault. The file checksum changed when I restored it from the backup. I'm not sure why it happened but I guess a bit error in 1 TB of data is something that can happen.

[Aaron Digulla]

My problem is that I don't have checksums for all files. So I need a way to validate those files, too.

[Aaron Digulla]

I do that, now. But that doesn't tell me if the file is already corrupt or not.

[Nathaniel Sharp]

@Aaron Digulla The link I provided above mentions "mencoder <filename.avi> -ovc lavc -oac lavc -o <output.avi>". Not sure if that works with your input files, though. You may have to experiment.

[schoppenhauer]

You might need to add -benchmark to the mplayer command line, to allow it to go faster than realtime.