Have xargs use alias instead of binary

Question

Bash 4.2 on CentOS 6.5:

In my ~/.bash_profile I have a bunch of aliases, including:

alias grep='grep -n --color=always'

so that I can get color highlighting and print line numbers automatically when running grep. If I run the following, highlighting works as expected:

$ grep -Re 'regex_here' *.py

However, when I ran this recently:

$ find . -name '*.py' | xargs grep -E 'regex_here'

the results were not highlighted and line numbers weren't printed, forcing me to go back and explicitly add -n --color=always to the grep command.

• Does xargs not read aliases in the environment?
• If not, is there a way to make it do that?

Solution 1

An alias is internal to the shell where it is defined. It is not visible to other processes. The same goes for shell functions. xargs is a separate application, which is not a shell, so doesn't have a concept of aliases or functions.

You can make xargs invoke a shell instead of invoking grep directly. However just invoking a shell isn't enough, you have to define the alias in that shell as well. If the alias is defined in your .bashrc, you can source that file; however this may not work your .bashrc performs other tasks that don't make sense in a non-interactive shell.

find . -name '*.py' | xargs bash -c '. ~/.bashrc; grep -E regex_here "$@"' _

Beware of the intricacies of nested quoting when typing the regexp. You can simplify your life by passing the regexp as a parameter to the shell.

find . -name '*.py' | xargs bash -c '. ~/.bashrc; grep -E "$0" "$@"' regex_here

You can perform the alias lookup explicitly. Then xargs will see grep -n --color=always.

find . -name '*.py' | xargs "${BASH_ALIASES[grep]}" regex_here

In zsh:

find . -name '*.py' | xargs $aliases[grep] regex_here

By the way, note that find … | xargs … breaks on filenames containing spaces (among others). You can fix this by changing to null-delimited records:

find . -name '*.py' -print0 | xargs -0 "${BASH_ALIASES[grep]}" regex_here

or by using -exec:

find . -name '*.py' -exec "${BASH_ALIASES[grep]}" regex_here {} +

Instead of calling find, you can do everything entirely inside the shell. The glob pattern **/ traverses directories recursively. In bash, you need to run shopt -s globstar to enable this glob pattern first.

grep regex_here **/*.py

This has a few limitations:

• If a lot of files match (or if they have long paths), the command may fail because it exceeds the maximum command line length.
• In bash ≤4.2 (but not in more recent versions, nor in ksh or zsh), **/ recurses into symbolic links to directories.

Another approach is to use process substitution, as suggested by MariusMatutiae.

grep regex_here <(find . -name '*.py')

This is useful when **/ isn't applicable: for complex find expressions, or in bash ≤4.2 when you don't want to recurse under symbolic links. Note that this breaks on file names containing spaces; a workaround is to set IFS and disable globbing, but it's starting to get a bit complex:

(IFS=$'\n'; set -f; grep regex_here <(find . -name '*.py') )

Solution 2

Use alias xargs='xargs '

alias: alias [-p] [name[=value] ... ]
(snip)
A trailing space in VALUE causes the next word to be checked for
alias substitution when the alias is expanded.

Solution 3

Please take this as a demonstration of another approach, which I cannot find in the the related SO question:

You can write a wrapper function for xargs which checks if the first argument is an alias and if so, expand it accordingly.

Here is a code which does exactly that but unfortunately it requires the Z shell and hence does not run 1:1 with bash (and frankly, I'm not used to bash enough to port it) :

xargs () {
        local expandalias
        if [[ $(which $1) =~ "alias" ]]; then
                expandalias=$(builtin alias $1) 
                expandalias="${${(s.'.)expandalias}[2]}"
        else
                expandalias=$1
        fi
        command xargs ${(z)expandalias} "${(z)@[2,-1]}"
}

Proof, that it works:

zsh% alias grep="grep -n"´                          # include line number of match
zsh% find foo -name "*.p*" | xargs grep -E test
foo/bar.p0:151:#data=test
foo/bar.p1:122:#data=test                           # line numbers included
zsh% unalias grep
zsh% find foo -name "*.p*" | xargs grep -E test
foo/bar.p0:#data=test
foo/bar.p1:#data=test                               # line numbers not included
zsh% 

Solution 4

A simpler, and more elegant solution, is to use process substitution:

grep -E 'regex_here' <( find . -name '*.py')

It does not create a new shell like the pipe does, which means you are still in your original shell where the alias is defined, and the output is exactly what you wish it to be.

Just be careful to leave no space between redirection and parenthesis, otherwise bash will throw an error. To the best of my knowledge, process substitution is supported by Bash, Zsh, Ksh{88,93}, but not by pdksh (I am told that should be a not yet).

View more solutions

Comments

[psimon]

This Q&A has what you want.

[MattDMo]

@psimon right, that's essentially saying to do what I already did in my workaround - I had to manually expand my alias in the xargs command. What I'm trying to find out is if there's a way that I can directly call my alias from xargs.

[MattDMo]

thank you for the clear explanation of why aliases aren't visible to other processes

[MariusMatutiae]

One may also use process substitution, see my answer.

[Gilles 'SO- stop being evil']

I think pdksh development is dead. Mksh is more or less a successor project — “pretty hard, it turns out (parsing concept is done in tg@’s head)”.

[Gilles 'SO- stop being evil']

But do not put --line-number or --color=always in GREP_OPTIONS unless it's just for one command, this will break a lot of scripts. --color=auto is ok to have there, and that's about all. Putting this line in your .bashrc will break a lot of stuff.

[doneal24]

@Gilles Setting any aliases or overriding default options for any command for the root account is a bad thing. Setting these options for a user account is unlikely to cause many problems. I cannot come up with any user scripts that are problematic.

[Gilles 'SO- stop being evil']

Just about any script that uses grep in a way that goes beyond testing the presence of an occurrence will break. For example, from /etc/init.d/cron on my system: value=`egrep "^${var}=" "$ENV_FILE" | tail -n1 | cut -d= -f2` . Or from /usr/bin/pdfjam: pdftitl=`printf "%s" "$PDFinfo" | grep -e … | sed -e …` . An alias is not a problem since it isn't seen in scripts.

[doneal24]

@Gilles I'm aware of many scripts like this. The ones I can't get around are generally run only by root (like /etc/init.d/cron). Personally, I don't have any aliases defined on my user accounts nor do I set options in a rc file or via environment variables to override the default behavior of commands. I prefer predictability over convenience.

[Gilles 'SO- stop being evil']

Aliases don't break predictability since they aren't seen by scripts. Setting GREP_OPTIONS breaks predictability very badly, except for a few options like --color=auto (which is what it was designed for).

[doneal24]

When you have accounts on some 800+ systems, not all of which share common file systems, keeping a consistent set of aliases is something I don't want to spend time doing. My fingers know how to type ls -aCF automatically without me having to think about it - no need for an alias.

[1.61803]

Np. It's also useful with sudo…