How to fix “No newline at end of file” warning for lots of files?

Question

I have a huge number of source files that are all lacking a newline at the end.

How do I automatically add a newline to the end of each of them?

Some may already have a newline, so it should only be added if necessary.

I'm probably not looking for code, per se, but just something I can run in Terminal to add the necessary newlines (or some kind of programming or development tool).

Solution 1

Converted Norman's answer to a split one-liner for convenience.

for i in * ; do  echo $i; \
 if diff /dev/null "$i" | tail -1 | \
  grep '^\\ No newline' > /dev/null; then echo >> "$i"; \
 fi; done

Replace * with whatever file pattern you want, eg *.c

And another to just tell you which files are broken:

for i in * ; do \
 if diff /dev/null "$i" | tail -1 | \
  grep '^\\ No newline' > /dev/null; then  echo $i; \
 fi; done

Solution 2

If you have access to Unix tools, you can run diff to find out which files lack a newline and then append it:

#!/bin/sh
for i
do
  if diff /dev/null "$i" | tail -1 | grep '^\\ No newline' > /dev/null
  then 
    echo >> "$i"
  fi
done

I'm relying on diff to produce the message with a \ in the first column, tail to give me the last line of diff's output, and grep to tell me if the last line is the message I'm looking for. If all that works, then the echo produces a newline and the >> appends it to the file "$i". The quotes around "$i" make sure things still work if the filename has spaces in it.

Solution 3

A simple fix for files that are "missing" newline at end of file is simply sed; the following fixes the file "in-place" (using the "-i" option):

find . -type f -exec sed -i -e '$a\' {} \; -print 

Explanation: find all files (-type f), run sed, change the files in-place (-i), given the following (-e) script/expression, which matches the end of the file ($), and perform the "append" action (a\), but don't actually specify any text to append (nothing after the \) which is going to add a newline to the end of the file, but only if it's missing. Prints all files found (fixed or not), which is probably unnecessary.

Main caveat is that sed features vary across platforms, so -i and -e may or may not be supported / the same; e.g. older Unix, or MacOS oddities may require slightly different syntax.

Solution 4

OK, after complaining in the comments, there is my better solution. First, you want to know, which files are missing newlines:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -v 0a$" ';' -print

Not super fast (calling a couple of processes for each file), but it's OK for practical use.

Now, when you have it, you may as well add the newline, with another -exec:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -v 0a$" ';' -exec sh -c "echo >> {}" ';'

Possible gotchas:

• if filenames are bad, e.g. they have spaces, you may need tail -1 \"{}\". Or does find do it right?

• you may want to add more filtering to find, like -name \*py, or the like.

• think about possible DOS/Unix newlines mess before use (fix that first).

EDIT:

If you don't like the output from these commands (echoing some hex), add -q to grep:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -q -v 0a$" ';' -print
find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -q -v 0a$" ';' -exec sh -c "echo >> {}" ';'

Solution 5

Try ex-way:

ex -s +"bufdo wq" *.c

And recursively (with a new globbing option enabled):

ex -s +"bufdo wq" **/*.c

This is equivalent to vi -es. Change *.c to extension of your interest.

The ex/vi would automatically append newline on save if it's not present.

View more solutions