How to remove blank lines from a file (including tab and spaces)?

Question

I want to remove all empty lines from a file. Even if the line contains spaces or tabs it should also be removed.

Solution 1

Just grep for non-blanks:

grep '[^[:blank:]]' < file.in > file.out

[:blank:], inside character ranges ([...]), is called a POSIX character class. There are a few like [:alpha:], [:digit:]... [:blank:] matches horizontal white space (in the POSIX locale, that's space and tab, but in other locales there could be more, like all the Unicode horizontal spacing characters in UTF8 locales) while [[:space:]] matches horizontal and vertical white space characters (same as [:blank:] plus things like vertical tab, form feed...).

grep '[:blank:]'

Would return the lines that contain any of the characters, :, b, l, a, n or k. Character classes are only recognised within [...], and ^ within [...] negates the set. So [^[:blank:]] means any character but the blank ones.

Solution 2

Here is an awk solution:

$ awk NF file

With awk, NF only set on non-blank lines. When this condition match, awk default action that is print will print the whole line.

Solution 3

How about:

sed -e 's/^[[:blank:]]*$//' source_file > newfile

or

sed -e '/^[[:blank:]]*$/d' source_file > newfile

i.e.

For each line, substitute:

• if it starts ("^")
• with spaces or tabs ("[[:blank:]]") zero or more times ("*")
• and then is the end of the line ("$")

More info on ::blank:: and other special characters at http://www.zytrax.com/tech/web/regex.htm#special

Solution 4

Looks like I've found one not that fast, but funny at last:

| xargs -L1

Solution 5

You can use sed command for removing blank lines:

sed '/^$/d' in > out

This command deletes all empty lines from the file "in"

View more solutions

Comments

[jordanm]

The wctype(3) and isalpha(3) manpages describe what the character classes will match.

[jordanm]

@MichaelDurrant It's not anchored on either side

[Stéphane Chazelas]

@MichaelDurrant. [^[:blank:]]$ would only match lines that end in a non-blank. We want lines that contain a non-blank anywhere

[Stéphane Chazelas]

You may want to remove the first one which doesn't answer the question.

[hamed golbahar]

@StephaneChazelas I tried grep [:blank:] SOURCEFILE even this command is working. I understand [] is for character class can you please give me some idea on how it works ? the snippet :blank: is new to me.

[hamed golbahar]

@MichaelDurrant can you please write some thing about [[:blank:]] ?

[Michael Durrant]

Added info for [[:blank::]]. Stephane, why doesn't the first work? I thought // at the end would replace the line without nothing.

[Mark Glossop]

Generally if I want to strip out blank lines as per OP, I'll use sed, but with the -i option to change "in place" (or "inline"). Worth reading the man page for the full caveat on using -i, but it saves on using a temporary output file if you have a string of inline text replacements that you need to make to a file.

[terdon]

Are there any cases where grep -E '\S' wouldn't work?

[Stéphane Chazelas]

@terdon \S is a perl regex operator, not a standard ERE one. I beleive it was also recently added to GNU regexp, but I don't expect many other implementations would support it. In any case, it is for space, not blank.

[terdon]

@StéphaneChazelas \S is for non-whitespace, not space. It matches any non-whitespace character. I thought it was PCRE only and just found out it works on normal GNU grep and sed on my Arch which is why I asked.

[Stéphane Chazelas]

@terdon I meant it's '[^[:space:]]' not [^[:blank:]]. posting on a smart phone where those characters are a pain to type. Check the opengroup site for the ERE syntax. The GNU implementation is generally not the one you want to try to test for portability as it's generally the one with the most extensions

[user1730706]

note bufdo is not POSIX pubs.opengroup.org/onlinepubs/9699919799/utilities/ex.html

[dave_thompson_085]

That doesn't delete lines containing only space and tab, as specifically requested.

[jringoot]

Nice short one, but it does more: remove leading spaces and tabs as well.

[poige]

Oh, it's found to be even more capable? — Nice! ;-P

[jringoot]

And it truncates lines (default at 1024 chars) See manpages: linux.die.net/man/1/xargs

[poige]

I'm falling deeper in love with that simple tool!!!11 )

[wisbucky]

Neat, this also removes lines with whitespace.

[AdminBee]

Welcome to the site, and thank you for your contribution. Unfortunately, strings only looks at "printable characters followed by a non-printable character". Space is "printable", so it will print lines that only contain spaces. The only reason why it apparently may work is because it has a lower threshold of what it considers a string (the default is 4 characters), so any line that contains less than 4 spaces and nothing else will be discarded - but so will be any line that contains less than 4 alphabetic characters. Lines with more than 4 spaces will be printed.

[Admin]

awk -i inplace NF file to replace the file.