Compare two file modification dates

Question

I am creating a generic compilation/transpilation system. One way to know if a file has been compiled/transpiled already would be to compare the source and target file modification dates.

I need to write a bash script that can do that:

source_file=foo;
target_file=bar;

stat_source=$(stat source_file);
stat_target=$(stat target_file);

but how can I extract the dates from the stat output and compare them? Is there a better way than stat to compare the most recent modification time of a file?

If I call stat on a log file, I get this:

16777220 12391188 -rw-r--r-- 1 alexamil staff 0 321 "Jun 22 17:45:53 2017" "Jun 22 17:20:51 2017" "Jun 22 17:20:51 2017" "Jun 22 15:40:19 2017" 4096 8 0 test.log

AFAICT, the time granularity is no finer than seconds. I need to get something more granular than that if possible.

Solution 1

In testing on this linux system. The usual way to test file times is the shell:

[ file1 -nt file2 ] && echo "yes"

Seems to work with seconds. This, which will touch the files with a time difference less than a second, doesn't detect that difference:

$ touch file2; sleep 0.1; touch file1; [ file1 -nt file2 ] && echo "yes"

To confirm the issue (time after the dot is nanoseconds):

$ ls --time-style=full-iso -l file?
-rw-r--r-- 1 user user 0 2017-06-23 01:37:01.707387495 -0400 file1
-rw-r--r-- 1 user user 0 2017-06-23 01:37:01.599392538 -0400 file2

The file1 is (a bit) newer than file2.

The problem now will be to correctly process the time value.

One solution is to use a formatted output of ls:

$ ls --time-style=+%s.%N -l file?
-rw-r--r-- 1 user user 0 1498196221.707387495 file1
-rw-r--r-- 1 user user 0 1498196221.599392538 file2

Extracting the time to two variables (without the dot):

$ file1time=$(ls --time-style=+%s%N -l file1 | awk "{print(\$6)}")
$ file2time=$(ls --time-style=+%s%N -l file2 | awk "{print(\$6)}")

And compare the times (times with nanoseconds just barely fit in a 64 bit value. If your system does not use 64 bit, this comparison will fail):

$ [ $file1time -gt $file2time ] && echo "yes"
yes

That shows that file1 is newer than file2

---

If ls fails to have the format needed, then you may try stat.

$ stat file1
  File: file1
  Size: 0               Blocks: 0          IO Block: 4096   regular file
Device: 805h/2053d      Inode: 9180838     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2017-06-23 01:37:01.707387495 -0400
Modify: 2017-06-23 01:37:01.707387495 -0400
Change: 2017-06-23 01:37:01.707387495 -0400
 Birth: -

If the output shows nanoseconds, the we will need date to parse (and format) the time.

$ stat --printf='%y\n' file1
2017-06-23 01:37:01.707387495 -0400

$ date +'%s%N' -d "$(stat --printf='%y\n' file1)" 
1498196221707387495

The rest is the same, assign the results of file1 and file2 to two variables and numerically compare them.

Solution 2

Given that you're using the stat (similar functionality, but different output format on BSDs and GNU), you could also use the test utility, which does this comparison directly:

   FILE1 -nt FILE2
          FILE1 is newer (modification date) than FILE2

   FILE1 -ot FILE2
          FILE1 is older than FILE2

In your example,

if [ "$source_file" -nt "$target_file" ]
then
    printf '%s\n' "$source_file is newer than $target_file"
fi

The feature is not available in POSIX (see its documentation for test), which provides as a rationale:

Some additional primaries newly invented or from the KornShell appeared in an early proposal as part of the conditional command ([[]]): s1 > s2, s1 < s2, str = pattern, str != pattern, f1 -nt f2, f1 -ot f2, and f1 -ef f2. They were not carried forward into the test utility when the conditional command was removed from the shell because they have not been included in the test utility built into historical implementations of the sh utility.

That might change in the future though as the feature is widely supported.

Note that when operands are symlinks, it's the modification time of the target of the symlink that is considered (which is generally what you want, use find -newer instead if not). When symlinks cannot be resolved, the behaviour between implementations (some consider an existing file as always newer than one that can't resolve, some will always report false if any of the operands can't be resolved).

Also note that not all implementations support sub-second granularity (bash's test/[ builtin as of version 4.4 still doesn't for instance, GNU test and the test builtin of zsh or ksh93 do, at least on GNU/Linux).

For reference:

• For the GNU test utility implementation (though note that your shell, if fish or Bourne-like, will also have a test/[ builtin that typically shadows it, use env test instead of test to bypass it), get_mtime in test.c reads struct timespec, and
• option -nt uses that data

Solution 3

GNU date can format the mtime of a file to be the nanoseconds since epoch:

date +%s%N --reference file

Those numbers are comparable with test or '['. It even works on bash v3.00 on an i386 (32-bit) system (circa 2008).

xxx=$(date +%s%N --reference file1)
yyy=$(date +%s%N --reference file2)
[ $xxx -lt $yyy ] && echo file1 is older than file2

Solution 4

If you're willing to assume non-embedded Linux, then you can use the test external command, which is part of GNU coreutils. (test is another name for [ and is a builtin in most shells). It has nanosecond granularity (up to the precision reported by the filesystem).

/usr/bin/test "$target" -nt "$source"

The -nt operator isn't defined by POSIX, but it's present in many implementations including dash, bash, pdksh, mksh, ATT ksh, zsh, GNU coreutils test, and BusyBox. However many implementations (dash, bash, pdksh, mksh, BusyBox — tested on Debian jessie) only support 1-second granularity.

But it would be a better idea to use tools dedicated to this job, such as make. Running a command only if a certain file is newer than some other file is the whole point of make. With the following content in a file called Makefile (note that you need a tab character before eacho command line).

target: source
    echo This command is only executed if target is newer than source
    do_stuff <source >$@

Run make target to execute the commands that generate it. If target exists and is newer than source, the commands are not executed. Read some documentation of make for more information.

Solution 5

POSIXly, you'd use find:

if find "$source_file" -prune -newer "$target_file" | grep -q '^'; then
  printf '%s\n' "$source_file is newer than $target_file"
else
  echo "It's not newer or one of the files is not accessible"
fi

For symlinks, that compares the mtime of the symlinks themselves. To compare the targets of the symlinks, add the -H or -L option.

That assumes $source_file doesn't start with - and doesn't otherwise correspond to one of the find predicates. If you need to deal with arbitrary file names, you'll need to do things like this first:

case $source_file in
  (["-+()!"]*) source_file=./$source_file;;
esac

GNU and FreeBSD find implementations at least do support sub-second granularity. AFAICT, macos doesn't seem to even store sub-second time information in the file attributes on the HFS+ file system at least.

View more solutions

Comments

[Alexander Mills]

thanks, will this give me a modified time that is more granular than seconds? about to find out...

[Alexander Mills]

my guess is that the modification time of files is no more granular than seconds, even with this tool?

[Marius]

According to the source-code, it compares struct timespec, which is potentially finer-grained (not all filesystems support subsecond timestamps, but you're likely to get what you wanted...)

[Alexander Mills]

nice this is mucho bueno...accept accept accept

[Alexander Mills]

seems to work on OSX sierra, but then again I installed some gnu tools on here I am sure at some point so...dunno for sure

[Marius]

BSDs have that option also - in answering, I checked and it didn't appear in POSIX (or I'd have mentioned that).

[Gilles 'SO- stop being evil']

@AlexanderMills The granularity depends on the implementation. Ksh93, zsh and GNU test have sub-second granularity, but bash, dash, mksh and BusyBox don't.

[Alexander Mills]

nice, yeah something more granular than seconds would be very handy. Can you add an example using make?

[Gilles 'SO- stop being evil']

@AlexanderMills I added an example but I do recommend that you read some documentation or tutotials. /usr/bin/test and shell builtins test implement mostly the same functionality but there can be minor diferences, and support for sub-second resolution of timestamps is one of these minor differences.

[Alexander Mills]

the date -r option seems to work, I guess that's the same as date--reference

[Alexander Mills]

I think date -r is probably a better option, and it appears to be x-platform