command: ls /etc | sort | grep d* is yielding no results but ls /etc | sort | grep p* lists entire directory

Question

I am playing around with piping and grep tonight. I know that grep uses regex and that * means 0 or more occurrences of the preceding character. So the way I understand it is that if I do the following command, the entire directory should be listed... but nothing is listed. All that happens is the command line resets:

[root@LinuxAcademy etc]# ls /etc | sort | grep d*
[root@LinuxAcademy etc]#

However, if I do the command again and replace d with p, the entire directory is listed just as I would expect:

[root@LinuxAcademy ~]# ls /etc | sort | grep p*
ConsoleKit
DIR_COLORS
DIR_COLORS.256color
DIR_COLORS.lightbgcolor
NetworkManager
X11
...........<<rest of listing not pasted in>>

Even though no error was given when i used the grep d*, I redirected stderr to a file just to check and nothing was printed.

I then thought that grep may be interpreting the 'd' in 'grep d*' to be an option or command... but that doesn't seem to be the case either.

Can anyone help me understand what is going on and why when i use the command with

grep d*

nothing is listed?

Solution

A regular expression is not a glob.

I am guessing that you want files that start with "d". In that case, you need:

ls /etc | sort | grep '^d'

What went wrong

How the following statement behaves depends on the files in the current directory:

ls /etc | sort | grep p*

Since it is unquoted, the shell will try to expand the glob p*, replacing it with whatever file names from the current directory match.

If no file names match, then a literal p* is passed to grep. Since grep expects regular expressions, it interprets p* to mean zero or more occurrences of the letter p. Since everything either has a p or doesn't, that matches everything.

Why would grep d* list nothing?

That likely means that there was a file name starting with d in the current directory. Just to be specific, let's say that file's name was dichotomy. The shell expands d* to dichotomy and passes that as the argument to grep. Thus, the effective command is:

grep dichotomy

That is likely why there were no results when you ran grep d*.

What happens if multiple files match the d* glob?

Consider the command:

grep d*

If their are multiple files matching that glob, say dmesg daemon.log and dpkg.log. The shell would expand d* to the list of those files in alphabetical order (which depends on your locale). This may result in:

grep daemon.log dmesg dpkg.log

This command searches for the presence of the string daemon.log in the files dmesg and dpkg.log. If that string does not occur in those files, no output results.

Comments

[Brad Harris]

That doesn't answer my question at all. I do really appreciate the explanation of processing order. What you said happens with p* is exactly what I expected. I also expect it when I use d*.... but nothing happens and that's the trivia here that I have. I am not trying with any specific purpose but to produce unexpected results and better understand the processing... which you did help me do so thank you. So why doesn't grep d* list entire directory just like grep p*. That's what I don't get.

[muru]

Oddly, the user's current directory is called etc in the first command, presumably /etc, so at least the directory whose name the glob was expanded into should have been matched. @BradHarris try this: After typing the command, press Tab - bash will try to autocomplete the d* glob. This will let you see what d* actually expands into before grep gets a hold of it.

[Brad Harris]

Oh man I didn't put 2 & 2 together when the processing path/order was explained when using p*. Thank you for the understanding. It's this understanding that I want to be sure I really get. I see it now.

[Brad Harris]

So I tested out the theory when grep d* is used. example: dmesg, daemon.log, and dpkg.log are the only files/dirs in the current directory that start with d. I copied those to the /etc directory. when I pass grep dmesg the result is dmesg. when i pass grep d*, therefore, it should at least print out a filename that starts with d because the shell would expand the d* to a name that is in both the current direction (and so was expanded to that name) but also in the directory being listed (/etc) and so grep would find it and return that. but nothing is returned still :(

[Brad Harris]

Ok, got it. Here's what's happening, when I was using d* it was trying to expand to a filename but there were multiple possible expansions and that would have been the failure I think. I tried a couple tests and when I use the * and the possible expansion is more than one, this behavior happens.

[John1024]

@BradHarris Yes, you got it. To explain in more words, I just added a section to the answer on the issue of multiple matches.