How can I remove all comments from a file?

Question

I have a file which includes comments:

foo
bar
stuff
#Do not show this...
morestuff
evenmorestuff#Or this

I want to print the file without including any of the comments:

foo
bar
stuff
morestuff
evenmorestuff

There are a lot of applications where this would be helpful. What is a good way to do it?

Solution 1

One way to remove all comments is to use grep with -o option:

grep -o '^[^#]*' file

where

• -o: prints only matched part of the line
• first ^: beginning of the line
• [^#]*: any character except # repeated zero or more times

Note that empty lines will be removed too, but lines with only spaces will stay.

Solution 2

I believe sed can do a much better job of this than grep. Something like this:

sed '/^[[:blank:]]*#/d;s/#.*//' your_file

Explanation

• sed will by default look at your file line by line and print each line after possibly applying the transformations in the quotes. (sed '' your_file will just print all the lines unchanged).
• Here we're giving sed two commands to perform on each line (they're separated by a semicolon).
• The first command says: /^[[:blank:]]*#/d. In English, that means if the line matches a hash at its beginning (preceded by any number of leading blanks), delete that line (it will not be printed).
• The second command is: s/#.*//. In English that is, substitute a hash mark followed by as many things as you can find (till the end of the line, that is) with nothing (nothing is the empty space between the final two //).
• In summary, this will run through your file deleting lines that consist entirely of comments and any lines left after that will have the comments stricken out of them.

Solution 3

As others have pointed out, sed and other text-based tools won't work well if any parts of a script look like comments but actually aren't. For example, you could find a # inside a string, or the rather common $# and ${#param}.

I wrote a shell formatter called shfmt, which has a feature to minify code. That includes removing comments, among other things:

$ cat foo.sh
echo $# # inline comment
# lone comment
echo '# this is not a comment'
[mvdan@carbon:12] [0] [/home/mvdan]
$ shfmt -mn foo.sh
echo $#
echo '# this is not a comment'

The parser and printer are Go packages, so if you'd like a custom solution, it should be fairly easy to write a 20-line Go program to remove comments in the exact way that you want.

Solution 4

Example input

cat example.sh

#!/bin/bash
# example script

echo "# test";# echo "# test"

# check the first parameter
if [ "$1" = "#" ]; then 
  # test couple of different cases
  echo "#"; # output # character 
  echo '\#'; # output # character '#' for test purpose
  echo \#\#\#; # comment # comment # comment '# comment'
  echo \#
  echo \#;
  echo \#; # comment
fi
# end of the script

Remove comments

sed -e '1{/^#!/ {p}}; /^[\t\ ]*#/d;/\.*#.*/ {/[\x22\x27].*#.*[\x22\x27]/ !{:regular_loop s/\(.*\)*[^\]#.*/\1/;t regular_loop}; /[\x22\x27].*#.*[\x22\x27]/ {:special_loop s/\([\x22\x27].*#.*[^\x22\x27]\)#.*/\1/;t special_loop}; /\\#/ {:second_special_loop s/\(.*\\#.*[^\]\)#.*/\1/;t second_special_loop}}' example.sh

Result

cat example.sh

#!/bin/bash

echo "# test";

if [ "$1" = "#" ]; then 
  echo "#"; 
  echo '\#'; 
  echo \#\#\#; 
  echo \#
  echo \#;
  echo \#;
fi

Read how it works at source: https://blog.sleeplessbeastie.eu/2012/11/07/how-to-remove-comments-from-a-shell-script/

Solution 5

You can achieve the required output using sed command. The below command had done the trick for me.

sed 's/#.*$//g' FileName

Where

• #.*$ - Regexp will filter all the string that starts with # up to the end of the line

Here we need to remove those lines so we replaced with empty so skipping 'replacement' part.

• g - mentioning repeated search of the pattern until end of file is reached.

General syntax of sed: s/regexp/replacement/flags FileName

View more solutions

Comments

[miracle173]

you cannot remove parts of a line with grep. you can use sed for this

[Anthon]

Your text and your example contradict. You write about lines being commented out, but clearly from the last line you mean line parts. And then the first line with a comment is deleted including EOL, and second second might be, but it is not clear as that is the last line. Please rephrase 'lines commented out' to be exact and disambiguate your examples.

[Арсений Черенков]

try using awk -F\# '$1!="" { print $1 ;} ' .

[Questionmark]

@Anthon I edited it like you said... :)

[Anthon]

@Questionmark what happens to the EOL after the comment on the last line? Can that be gobbled up? (Leaving the file ending in a line without a newline).

[Questionmark]

@Anthon Sure... That works

[Questionmark]

@Kusalananda Interesting. Hadn't thought about it. According to all the answers that have been posted, I think we would end up with just echo '. Do you have a clever way to handle those?

[Kusalananda]

@Questionmark I might be clever, but I'm not writing-a-shell-grammar-parser clever.

[Devy]

@Archemar your awk answer is the BEST out of all the other answer here including the sed/grep/egrep etc.

[Арсений Черенков]

@Devy thanks, this is but a dirty hack, it doesn't take into account kusalanada's comment. (and after 4 1/2 years I am not turning it to a answer)

[Devy]

@Archemar not turning it to an answer is fine. And 90% of the use cases here is to remove comment lines (not trailing comments) so that default config files are more concise and readable. I am sure a lot of readers would agree.

[Eric McLachlan]

grep -v '^#.*' filename (Sorry, I can't submit my own answer.) (-v inverts the match, so it returns all rows that DON'T match a line starting with #.)

[Raza]

@alinh Thanks for reviewing the answer. Please note that the question required not only the beginning of the line but anywhere in the file. This is also showing in his/her expected result in the question above. My answer would be incorrect if I only look for beginning of the line.

[alinh]

zzz. my bad, didn't see the last line :(

[Jeff Hewitt]

This will completely remove the line starting with evenmorestuff in the OP's example.

[αғsнιη]

note: 4th line replaced with new line in this case.

[Raza]

@JosephR. good catch. I missed that earlier. In this case grep -o '^[^#]*' file would be the best solution. this is already explained by jimmij. thanks for your review

[WestCoastProjects]

It will also delete anything found after a hash inside a string, no ? E.g. mystring="Hello I am a #hash" will become mystring="Hello I am a"

[Basile Starynkevitch]

I would use grep -v '^#' file > newfilewithoutcomments

[Kusalananda]

Try that with a script containing that sed command...

[Wildcard]

It should be noted this is NOT a general method for shell scripts, as for example the line somvar='I am a long complicated string ## with special characters' # and I am a comment will not be handled correctly.

[Wildcard]

@javadba, yes, but at that point you might as well use a full parser. What's going to be using this data that can understand quotes and variable assignments but can't handle comments? (This is why many config files such as crontab only allow full-line comments, with or without leading whitespace, but do not allow trailing comments on a line. The logic is MUCH simpler. Use only the first of the two Sed instructions in this answer for a crontab comment stripper.)

[Pierz]

This variant works better for me (on a Mac): grep -o '^[^#].*' file

[JBallin]

The comments are gone but I'm seeing a bunch of white space in their place in the output? sed solution only has one blank line, seems like a solid argument to use other answer, unless I'm missing something?

[jimmij]

@JBallin Did you define some alias for grep maybe? Try changing grep to command grep, if you still see spaces post the sample input.

[JBallin]

@jimmij gist.github.com/JBallin/c89829846df419d7604384228a305d5a

[jimmij]

@JBallin Cannot reproduce that, what version is it (grep --version)?

[JBallin]

@jimmij looks like I'm using the Mac built-in (not 3+): grep (BSD grep) 2.5.1-FreeBSD

[jimmij]

@JBallin I'm not sure why it behaves so weirdly on FreeBSD. You can try to add -P option (perl compatible mode), if it is available on your system.

[JBallin]

@jimmij no dice. usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...]

[RBF06]

great answer, this looks like a great balance of utility vs. complexity for a wide array of general use-cases, but in the case that you know ahead of time that you only need to delete lines starting directly with # (in column 1), is there any benefit to sed over grep -v "^#" ?

[Ray Butterworth]

How did this get 40 upvotes and become selected as the best answer??? It doesn't even handle the simple case print "#tag" # Print a hashtag..

[Ray Butterworth]

It won't handle print "#tag" # Print a hashtag.

[Ray Butterworth]

It won't handle print "#tag" # Print a hashtag.

[Ray Butterworth]

It won't handle print "#tag" # Print a hashtag.

[Ray Butterworth]

It won't handle print "#tag" # Print a hashtag.

[Ray Butterworth]

It won't handle print "#tag" # Print a hashtag.

[jackbmg]

Ah, right... of course. Thanks for pointing that out. I was looking for an answer with regards to typical linux configuration files, such as pam.d configs, so I didn't think of that. I guess it would have to be adapted to find and remove any comments that lie on the same line as code. I just saw probably a better solution to my particular issue above: egrep -v "#|$^"

[Eduardo Cuomo]

It doesn't even handle the simple case print "#tag" # Print a hashtag..

[Popup]

That's the way to do it!

[Popup]

That's by far the best solution so far, but I even this fails on multi-line strings with hashes in them. echo -e "this is \n#NOT# a comment" I have no idea of how to solve that, though.

[Popup]

Like most answers, this will even kill the shebang!

[Popup]

Like most answers, this will even kill the shebang!

[Jimmy Adaro]

This would screw your script if you're using something like: EXTENSION="${FILENAME##*.}" (it'll delete ##*.})

[Ray Butterworth]

@KarnKumar, the question explicitly says that evenmorestuff#Or this should be stripped to evenmorestuff.

[Vincent Yin]

A small enhancement... sed '/^[[:blank:]]*#/d;s/[[:blank:]]*#.*//' your_file. Your original command leaves some trailing blanks in a line. My enhancement gets rid of those, too.

[Eduardo Cuomo]

It is the wrong answer!

[Admin]

sed: 1: "1{/^#!/ {p}}; /^[\t\ ]* ...": extra characters at the end of p command

[Admin]

And actually, it's full of holes. It would cut echo "#\"#" as a comment.