Prune duplicate entries from PATH variable

Question

I modify my .bashrc frequently and then source it. However, when I have things like export PATH="~/bin:~/perl5/bin:$PATH" in my file, then the PATH environment variable grows every time I source the file.

For example, the first time .bashrc is sourced, the PATH variable consists of ~/bin:~/perl5/bin:/usr/bin:/bin.

The second time it consists of ~/bin:~/perl5/bin:~/bin:~/perl5/bin:/usr/bin:/bin.

The third time it consists of ~/bin:~/perl5/bin:~/bin:~/perl5/bin:~/bin:~/perl5/bin:/usr/bin:/bin.

Is there a simple way to make it only add anything that isn't already in the PATH?

Solution 1

Use the pathmunge() function available in most distro's /etc/profile:

pathmunge () {
if ! echo $PATH | /bin/egrep -q "(^|:)$1($|:)" ; then
   if [ "$2" = "after" ] ; then
      PATH=$PATH:$1
   else
      PATH=$1:$PATH
   fi
fi
}

edit: For zsh users, typeset -U <variable_name> will deduplicate path entries.

Solution 2

I was having this issue so I used a combination of techniques listed on StackOverflow question. The following is what I used to dedupe the actual PATH variable that had already been set, since I didn't want to modify the base script.

    tmppath=(${PATH// /@})
    array=(${tmppath//:/ })
    for i in "${array[@]//@/ }"
    do
        if ! [[ $PATH_NEW =~ "$i" ]]; then
            PATH_NEW="${PATH_NEW}$i:";
        fi
    done;
    PATH="${PATH_NEW%:}"
    export PATH
    unset PATH_NEW

You could always optimize this a bit more, but I had extra code in my original to display what was happening to ensure that it was correctly setting the variables. The other thing to note is that I perform the following

1. replace any SPACE character with an @ character
2. split the array
3. loop through the array
4. replace any @ characters in the element string with a space

This is to ensure that I can handle directories with spaces in (Samba home directories with Active Directory usernames can have spaces!)

Solution 3

I can think of two different ways you could resolve this. The first one, is to start your .bashrc with a line that explicitly sets your base PATH, that way every time you source it, it is reset to the base prior to adding additional directories.

For example, add:

# Reset the PATH to prevent duplication and to make sure that we include
# everything we want.
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Alternately, you can check for an item before you add it to the path. To do that, you'd use something like:

if ! [[ $PATH =~ '~/perl5/bin' ]]
then
    PATH="~/perl5/bin:$PATH"
fi

The latter tends to get a little repetitive if you're adding a lot of entries, however, so I tend to stick with the former. If you wanted to use this and planned on adding a lot of entries, writing a bash function to handle it would be wise.

Note: The second option may only work as written in modern versions bash. The regular expression support is not a Bourne Shell (/bin/sh) feature, and may not exist in other shells. Also, the use of quotes may not be needed or may even cause problems on some newest versions of bash.

Solution 4

Only one string:

for i in $(echo $PATH|tr ":" "\n"|sort|uniq);do PATH_NEW="${PATH_NEW}$i:";done;PATH="${PATH_NEW%:}"

Solution 5

Set your path explicitly.

View more solutions

Comments

[Christopher Bottoms]

Thanks! It's not in /etc/profile on Debian Lenny, so I include it in my .bashrc.

[Christopher Bottoms]

Thanks. I tried setting the path explicitly, but I have a .bashrc file that I use in multiple environments, so the exact default PATH is not always the same.

[demonkoryu]

You can still handle that by checking in the script to see what your local hostname is, and then setting your absolute path appropriately for that server.

[Christopher Bottoms]

Thanks. I tried setting the path explicitly, but I have a .bashrc file that I use in multiple environments, so the exact default PATH is not always the same.

[Christopher Bottoms]

Thanks. One difficultly this would cause me is that it alphabetically (or ASCIIbetically) reorders the contents of $PATH. I like to put specific directories at the beginning of $PATH (like /home/username) so that my personal copies of executables are run instead of the built-in defaults.

[demonkoryu]

Cleaner way of doing the check to see if the hew directory exists in the current path, in modern bash shells: if ! [[ $PATH =~ (^|:)$1($|:) ]] ; then.

[13ren]

Just a typo: the ! is missing in the if. Also, it's surprisingly (to me) easy to make this into a function which loops over space-separated arguments, so you can just say: add_to_PATH ~/perl5/bin ~/.bin unix.stackexchange.com/a/4973/28760 (I guess the case statement used there is more portable, but your if is clearer to me). EDIT weird coincidence that that question also adds perl5!

[demonkoryu]

@13ren : Thanks for the note. I also just realized I used single quotes in the PATH set line, instead of double-quotes like I should have. As written, it would have replaced your existing path with the variable name. Oops! Apparently that was a bad entry for me for typos; both are fixed now.

[Don Hatch]

I wouldn't use this. If my naive version said PATH=/foo:$PATH that means I want /foo to win. pathmunge /foo before doesn't accomplish that. A better way would be to add /foo at the beginning and then remove duplicates afterwards.

[Sukima]

WARNING: This implementation has a major bug!! it will remove /bin if there are other entries like /usr/bin because the regexp matches even though the two entries are not the same!

[Sukima]

Here is an alternative implementation that fixes that bug: github.com/sukima/dotfiles/blob/…