How to test if string exists in file with Bash?

Question

I have a file that contains directory names:

my_list.txt :

/tmp
/var/tmp

I'd like to check in Bash before I'll add a directory name if that name already exists in the file.

Solution 1

grep -Fxq "$FILENAME" my_list.txt

The exit status is 0 (true) if the name was found, 1 (false) if not, so:

if grep -Fxq "$FILENAME" my_list.txt
then
    # code if found
else
    # code if not found
fi

Explanation

Here are the relevant sections of the man page for grep:

grep [options] PATTERN [FILE...]

-F, --fixed-strings

        Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.

-x, --line-regexp

        Select only those matches that exactly match the whole line.

-q, --quiet, --silent

        Quiet; do not write anything to standard output. Exit immediately with zero status if any match is found, even if an error was detected. Also see the -s or --no-messages option.

Error handling

As rightfully pointed out in the comments, the above approach silently treats error cases as if the string was found. If you want to handle errors in a different way, you'll have to omit the -q option, and detect errors based on the exit status:

Normally, the exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless the -q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore advisable, for the sake of portability, to use logic that tests for this general condition instead of strict equality with 2.

To suppress the normal output from grep, you can redirect it to /dev/null. Note that standard error remains undirected, so any error messages that grep might print will end up on the console as you'd probably want.

To handle the three cases, we can use a case statement:

case `grep -Fx "$FILENAME" "$LIST" >/dev/null; echo $?` in
  0)
    # code if found
    ;;
  1)
    # code if not found
    ;;
  *)
    # code if an error occurred
    ;;
esac

Solution 2

Regarding the following solution:

grep -Fxq "$FILENAME" my_list.txt

In case you are wondering (as I did) what -Fxq means in plain English:

• F: Affects how PATTERN is interpreted (fixed string instead of a regex)
• x: Match whole line
• q: Shhhhh... minimal printing

From the man file:

-F, --fixed-strings
    Interpret  PATTERN  as  a  list of fixed strings, separated by newlines, any of which is to be matched.
    (-F is specified by POSIX.)
-x, --line-regexp
    Select only those matches that exactly match the whole line.  (-x is specified by POSIX.)
-q, --quiet, --silent
    Quiet; do not write anything to standard output.  Exit immediately with zero status  if  any  match  is
          found,  even  if  an error was detected.  Also see the -s or --no-messages option.  (-q is specified by
          POSIX.)

Solution 3

Three methods in my mind:

1) Short test for a name in a path (I'm not sure this might be your case)

ls -a "path" | grep "name"

2) Short test for a string in a file

grep -R "string" "filepath"

3) Longer bash script using regex:

#!/bin/bash

declare file="content.txt"
declare regex="\s+string\s+"

declare file_content=$( cat "${file}" )
if [[ " $file_content " =~ $regex ]] # please note the space before and after the file content
    then
        echo "found"
    else
        echo "not found"
fi

exit

This should be quicker if you have to test multiple string on a file content using a loop for example changing the regex at any cicle.

Solution 4

Easiest and simplest way would be:

isInFile=$(cat file.txt | grep -c "string")


if [ $isInFile -eq 0 ]; then
   #string not contained in file
else
   #string is in file at least once
fi

grep -c will return the count of how many times the string occurs in the file.

Solution 5

Simpler way:

if grep "$filename" my_list.txt > /dev/null
then
   ... found
else
   ... not found
fi

Tip: send to /dev/null if you want command's exit status, but not outputs.

View more solutions

Comments

[Noam Manos]

To find all of the strings inside a file, you can run grep in FOR loop: unix.stackexchange.com/a/462445/43233

[Toren]

If I execute this command from bash script how to catch 0 or 1 into a variable ?

[Shawn Chin]

@Toren Most recent exit status can be accessed using $?. you can also use the grep command alongside the if statement (as shown in updated answer).

[SourceSeeker]

You can use grep -Fqx "$FILENAME" and you don't have to worry about regex characters in the variable contents and you won't have to use them in the search string.

[Keyur Padalia]

Ah, good one! Didn't know about -x (and -F by itself was obviously not enough). Edited.

[sorpigal]

You don't need to test the output of grep, you can just use grep -q and call grep directly from if as Thomas does in his answer. In addition, the question didn't include checking whether the directory exists before adding it to the list (it could be a list of deleted directories, after all).

[Toren]

Can you please provide some example of using grep -Fqx ? I didn't get it what it good for . 10x

[Keyur Padalia]

@Toren: It's already there in my answer. Or isn't that what you meant?

[Toren]

@Thomas :I understand why use -q , but can you add some more explanation why to use -Fx . I do not understand it from your comment or from grep manual. Thank you in advance.

[Keyur Padalia]

@Toren: Without -F, your filename would be interpreted as a regular expression, not as a normal string. Thus, if it contains strange characters (like .), these will be interpreted by grep's regex engine, and not matched literally. The -x flag ensures that the string matches an entire line, not just a part of it: so if you search for foo, only lines containing exactly foo are matched, not foobar. (There was an error in my examples; it still used some regex syntax. Fixed now.)

[lecodesportif]

I removed the example script, it didn't add anything to the answer given by Thomas.

[Adam S]

-F does not affect the file processing, it affects how PATTERN is interpreted. Typically, PATTERN is interpreted as a regex, but with -F it will be interpreted as a fixed string.

[wonder.mice]

To be good when file not exists and avoid getting errors in stderr use if grep -OPTIONS "<needle>" file.txt &> /dev/null; then

[Keyur Padalia]

You'd probably want to know that that happened (e.g. if you ran the script from the wrong directory), instead of silently giving the wrong answer.

[EminezArtus]

Why are the spaces necessary before and after the $file_contenet?

[Aaron R.]

This even works in the limited /bin/sh offered by VMware's ESXi 5.5 :)

[Keyur Padalia]

Probably grep isn't part of /bin/sh but a separate program. All flags in use here are part of POSIX so you can expect them to be available either way.

[B T]

How can you do this with a variable? If i store the result of grep in a variable, I can't get it to work.

[Keyur Padalia]

@BT: grep ..., then assign like grep_status=$? and test it with if [[ $grep_status -eq 0 ]].

[andrej]

I have to swap found and not found branches for this to work

[Keyur Padalia]

@andrej: That's really weird, and suggests that your grep is not POSIX compliant (see here). What OS/distro/grep are you using? What does grep -Fxq foo <(echo foo); echo $? output (I get 0)? How about grep -Fxq foo <(echo bar); echo $? (I get 1)?

[andrej]

@thomas 0 and 1 as for you. I have to doublecheck my previous check.

[Schmick]

A couple of notes for folks looking at this answer: 1) In bash, 0 is always true and anything else is always false 2) Only use the -x flag if you want the entire line to match exactly. If you just want to find if your string exists in the file at all, leave that off. If you want to find if your string exists exactly but without matching an entire line necessarily (i.e., as a whole word), use -w.

[Nam G VU]

I don't see -c option in fgrep --help

[codeforester]

Uses too much memory in case the file is large. grep -q described in the accepted answer is the most efficient approach.

[rogerdpack]

or use -q which is same as --quiet :)

[tatsu]

agree on the -q also best answer here, and is fourth place. no justice in this world.

[Wassadamo]

Plus 1 for the fast solution and a more generalizable one

[redanimalwar]

I do not gethe the -q / --silent is it needed? Its falsely says "all good" to bash even if a error occurs. If I got that correctly. Seems a flawed concept for this case.

[redanimalwar]

I am still confused why not just do it without -q. I do not want errors to be ignored, I also do not want a complicated case statement. What is so bad in having the grep output in a script? Seems stupid to me that it just ignores errors. From the man: Portable shell scripts should avoid both -q and -s and should redirect standard and error output to /dev/null instead. (-s is specified by POSIX.) But here you and the other guys give people advice to do exactly that!

[Keyur Padalia]

@redanimalwar In my edited version at the bottom of this post, I'm doing exactly that (do it without -q and avoid both -q and -s and should redirect standard and error output to /dev/null instead). The problem is fundamental: there are three possible outcomes (found, not found, error) so how can you possibly expect to handle it with a single if statement? Instead of the complicated case statement you can use two ifs if you prefer, but I think the case is easier to read.

[Shinoy Shaji]

@ThomWiggers I tried the same and it worked for me.

[Hashim Aziz]

What is the =~ and where can I learn more about it?

[Edenshaw]

I would recommend adding -q so that you don't get the found strings in the stdout

[mightypile]

@HashimAziz =~ is bash's regex operator. Try stackoverflow.com/a/19441575/2846766 or search for "bash shell regex operator".