keep duplicates out of $PATH on source

Question

I have the following code that's source-d by my .shellrc

PATH="${PATH}:${HOME}/perl5/bin"
PATH="${PATH}:${HOME}/.bin"
export PATH

but if I make changes to other code and then source this file, my path continues to get longer and longer with each source, each time appending these when they're already there. What can I do to prevent this?

Solution 1

add_to_PATH () {
  for d; do
    d=$({ cd -- "$d" && { pwd -P || pwd; } } 2>/dev/null)  # canonicalize symbolic links
    if [ -z "$d" ]; then continue; fi  # skip nonexistent directory
    case ":$PATH:" in
      *":$d:"*) :;;
      *) PATH=$PATH:$d;;
    esac
  done
}
add_to_PATH ~/perl5/bin ~/.bin

The line for symbolic link canonicalization is optional. If you remove it, also remove the next line (if you want to keep nonexistent directories), or change it to

if ! [ -d "$d" ]; then continue; fi

Note that the symlink canonicalization method only guarantees unicity amongst directories that were added by this function. It also doesn't handle edge cases like an NFS directory mounted on two locations or a Linux bind mount.

Solution 2

You could put a test around the "append this directory to path" command which would check to see if foo is already in the path before adding it, but it wouldn't buy you much.

First, the test itself would be costly compared to appending a duplicate element. Secondly, a redundant element later in the path has no effect upon what does get executed when you execute a given command because the first matching executable in the path will still be the one executed. Finally most shells cache prior path hits in a hash table so the second time you execute my_command the path isn't even searched.

About the only thing that not appending redundant entries will get you is a prettier looking path, but most paths are pretty ugly to begin with. If this aesthetic goal is really important to you, tell us which shell you are using and I can conjure up a function to "append this to path only if it isn't present" function.

Solution 3

I use these functions that are sourced from an initialization script by fink on os x (so credit goes to the fink developers). They work great and I can re-source my .bash_profile whenever I want. Don't ask me how they work... I just know they do :)

# define append_path and prepend_path to add directory paths, e.g. PATH, MANPATH
# add to end of path
append_path()
{
  if ! eval test -z "\"\${$1##*:$2:*}\"" -o -z "\"\${$1%%*:$2}\"" -o -z "\"\${$1##$2:*}\"" -o -z "\"\${$1##$2}\"" ; then
    eval "$1=\$$1:$2"
  fi
}

# add to front of path
prepend_path()
{
  if ! eval test -z "\"\${$1##*:$2:*}\"" -o -z "\"\${$1%%*:$2}\"" -o -z "\"\${$1##$2:*}\"" -o -z "\"\${$1##$2}\"" ; then
    eval "$1=$2:\$$1"
  fi
}

I can use them like so to append or prepend to $PATH or $MANPATH (they'll work with any variable formatted like $PATH):

prepend_path PATH $macPortsDir/sbin
prepend_path MANPATH $macPortsDir/man

Comments

[Admin]

See also: How do I manipulate $PATH elements in shell scripts? on stack overflow.

[Admin]

superuser.com/questions/39751/…

[xenoterracide]

would it really end up being shell specific? :( I'm trying to keep this file shell agnostic

[Gilles 'SO- stop being evil']

A redundant path does mean slower traversals sometimes. You can feel it when you have path directories on NFS and you mistype a command name or your shell's cache isn't aggressive enough.

[msw]

This is close to a shell agnostic idempotent append, but even this snippet will yield ~/perl5/bin:~/.bin:~/.bin when run twice because of the lack of trailing : from the first run. And you can't add a trailing : because the empty component thus generated implies ..

[msw]

@Gilles, agreed. But, as noted in my comment to your answer, the semantics of null PATH components make this problem harder than it appears at first.

[msw]

Also, twiddle ~ expansion and symlinks make this problem nearly insoluble in the general case.

[Gilles 'SO- stop being evil']

~ expansion has nothing to do with this. If you want to eliminate a component that's a symlink to another component, that's a different problem, but it's solvable (with the caveat that symlinks can change over time).

[Gilles 'SO- stop being evil']

@msw: No, this snippet will yield /usr/local/bin:/usr/bin:/bin:/home/xenoterracide/perl5/bin:/‌​home/xenoterracide/.‌​bin if run twice (assuming PATH is /usr/local/bin:/usr/bin:/bin initially). Note that I'm matching :$PATH:, not $PATH.

[msw]

I did indeed miss the additional colons; you are correct. +1

[xenoterracide]

since I didn't expect to get a loop... if I'm only doing path at a time... ~/perl5 doesn't exist on all systems... so what's the best way to do this if I'm only doing one at a time? either that or what's the best way to have this code test for directory existence before adding them to the path

[jlettvin]

I really like the symbolic link hack. I will use it in my own .bashrc. Here is a simple pair of easy-to-read functions which do the job without that hack. I wish comments allowed newlines. ____________________________________________________________ append_path() { ((echo ${PATH} | tr ':' '\n'|grep -c "$1")) || export PATH=${PATH}:$1 } ____________________________________________________________ prepend_path() { ((echo ${PATH} | tr ':' '\n'|grep -c "$1")) || export PATH=$1:${PATH} }

[vijay]

@Giles What error is guarded against by including the || pwd part?

[Gilles 'SO- stop being evil']

@jrw32982 Ancient or bizarre shells that don't support pwd -P. With pwd as a fallback, the script will still work, it just won't detect symbolic links.