How to print a file, excluding comments and blank lines, using grep/sed?

Question

I'd like to print out a file containing a series of comments like:

    </Directory>
    ErrorLog ${APACHE_LOG_DIR}/error.log
    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
    #   SSL Engine Switch:

In essence, the file contains multiple indentation levels, where a comment starts with a # symbol.

grep should remove blank lines, and also lines where there is a hash symbol before text (implying that these are comments).

I know that blank lines can be deleted via: grep -v '^$'

However how can I delete lines with leading whitespace, and then a # symbol, and print out only lines with actual code? I would like to do this in bash, using grep and/or sed.

Solution 1

With grep:

grep -v '^\s*$\|^\s*\#' temp

On OSX / BSD systems:

grep -Ev '^\s*$|^\s*\#' temp

Solution 2

With awk:

awk '!/^ *#/ && NF' file

• !/^ *#/ - means select lines which do not have space followed by # symbol
• NF - means select lines which are not blank
• && - is an and operator which ensure if both of the above are true then print the line.

Solution 3

This is probably easier with sed than grep:

sed -e '/^[[:space:]]*$/d' -e '/^[[:space:]]*#/d' test.in

Or with an ERE:

# Gnu sed need -re instead of -Ee
sed -Ee '/^[[:space:]]*(#|$)/d' test.in

With the ERE, grep can do it fairly easily too:

# Not sure if Gnu grep needs -E or -r
grep -vE '^\s*(#|$)' test.in

# or a BRE
grep -v '^\s*\(#\|$\)' test.in

Solution 4

Code for GNU sed:

sed -r '/^(\s*#|$)/d;' file

---

$cat file
    </Directory>
    ErrorLog ${APACHE_LOG_DIR}/error.log
    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    next line is empty

       line with leading space
       # line with leading space and #
    LogLevel warn
        #line with leading tab and #
        line with leading tab
    CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
    #   SSL Engine Switch:
$sed -r '/^(\s*#|$)/d;' file
    </Directory>
    ErrorLog ${APACHE_LOG_DIR}/error.log
    next line is empty
       line with leading space
    LogLevel warn
        line with leading tab
    CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined

Solution 5

Little Upgrade

grep -v '^\s*$\|^#\|^\s*\#' filename

This code excludes empty lines or lines with only spaces, lines beginning with #, and lines containing nothing but spaces before #.

PS: ^# is different than ^\s*#

View more solutions

Comments

[Joel G Mathew]

Is this the simplest form?

[Kevin]

You can do it somewhat nicer with an extended regex, see my edit.

[Saghir A. Khatri]

plz explain your answer in simple words, instead of posting the code only

[RobEarl]

This is really the same as the accepted answer. ^# is redundant as ^\s*# matches zero or more whitespace followed by a hash.

[Francisco Cabrera]

Show me only eXecutABLEs lines ( xable.sh )

[Francisco Cabrera]

Show me only eXecutABLEs lines

[Nathan Tuggy]

Welcome to Stack Overflow! Generally, answers are much more helpful if they include an explanation of what the commands are intended to do, and why that solves the problem without introducing others. (This post was flagged by at least one user, presumably because they thought an answer without explanation should be deleted.)

[draeath]

This doesn't seem to catch cases where the comments have leading whitespace, keep in mind.

[Nikhil Gupta]

It will not work if there are lines which are having only white spaces (space, tab etc). Below worked for me. egrep -v '^[[:space:]]*$|^ *#' <file_name>

[pipe]

Would you care to explain how ^# is different from ^\s*#?

[NeronLeVelu]

egrep -v '^[[:blank:]]*#|^[[:blank:]]*$' <file_name> newline, vertical tab, form feed, and carriage return are not part of a line in grep (by default) and blank could occur in empty lines

[Chris Stryczynski]

Why is the | being escaped? This did not work for me on ZSH. Instead I had to do: grep -v "^$|^\s*\#"

[gbrener]

@ChrisStryczynski, you may not need to escape the |, however I suspect this has more to do with your version of grep (e.g. BSD vs GNU) than your choice of shell.

[Chris Stryczynski]

Apologies, it does actually work for me. I forgot I had ripgrep aliased to grep.

[edib]

This does not work on /etc/apache2/sites-enabled/000-default.conf

[Christophe Roussy]

This also does not handle leading whitespace ` #`, which technically is a comment too.

[Ernesto Iser]

@ChristopheRoussy it does handle leading whitespaces, the ^\s+$ part takes care of that

[Christophe Roussy]

but not leading whitespace + comment after that ?

[Ernesto Iser]

@ChristopheRoussy I see what you are saying, you are right, working on a fix, thanks for pointing that out

[Ernesto Iser]

this is a very bad use of the cat command, you could just use egrep without the cat

[Mintaka]

^# -> only match hastag letter on beginning of the line <br/> ^\s*# -> also match hastag letter with spaces before

[Paul Kenjora]

Did not work on OSX, removed comments but not newlines.

[gbrener]

Updated the answer to cover whitespace-only lines and OSX/BSD systems.

[Joel G Mathew]

That wont work when there are spaces beforce the comment characters.

[citysurrounded]

It's redundant. /s* means "match 0 or more spaces". Thus ^\s*# would also match ^#.

[MaXi32]

This is what I'm looking for. Your solution is the only one work to remove CSF # do not delete string that also has another pound sign between, so it will only output an IP addrress. Thanks for providing this alternative answer