Best Practices with Anaconda and Brew

Question

I have just got a new Macbook with OSX Sierra, so want to ensure my development environment is setup properly.

I am looking to follow the 'best practices' mentioned here: https://github.com/nicolashery/mac-dev-setup

I need Python 2.x for work (urllib, Pandas, Numpy, Scikit-learn), and Python 3.x for some online classes (Pandas, Numpy, Django) I am taking. I have installed Python 2 and 3, using brew install python and brew install python3 respectively.

However, on this link, there is no mention of Anaconda, just IPython. Given that I already have Python 2 and 3 installed via Homebrew, is it even advisable to use anaconda, or should I stick to standard IPython as mentioned on the Github link above? I am confused after reading this post: OS X - Deciding between anaconda and homebrew Python environments

If Brew and Anaconda can indeed work together, what specific steps can I take to ensure that there are no conflicts between the two versions?

Solution 1

brew and conda tend not to play nicely together, but I think I have a set up that has worked quite well for me so far. It was inspired by this post.

You can add the following code to your .zshrc:

# Deactivates conda before running brew. 
# Re-activates conda if it was active upon completion.

brew() {
    local conda_env="$CONDA_DEFAULT_ENV"
    while [ "$CONDA_SHLVL" -gt 0  ]; do
        conda deactivate
    done
    command brew $@
    local brew_status=$?
    [ -n "${conda_env:+x}" ] && conda activate "$conda_env"
    return "$brew_status"
}

You want to deactivate conda before running brew so that brew doesn't find conda packages in your PATH whenever it tries to install something. In fact, brew doctor will complain if you have not deactivated conda before running brew, as mentioned in the post I link to above. (See also this question.)

One thing I should mention is that conda environments "stack", but the brew() function I've written above does not keep track of your stack of environments. (See below for a version of this function that keeps track of this.) For example, if you do conda activate newenv while a conda environment oldenv is active, then conda deactivate will return you to oldenv. However, if you run brew using the function I've written above after activating oldenv and then newenv, running conda deactivate will not return you to oldenv but will deactivate your conda environments entirely.

This function also probably creates some unnecessary overhead when running brew, as I believe you only really need to deactivate your conda environment when running brew install. That said, if you're the kind of person to care about that overhead enough, this answer probably doesn't tell you anything you didn't already know.

As a final note, brew cask install anaconda does not strike me as a good idea, since conda was designed to be installed in $HOME, but brew cask will want to install it in /usr/local, so that could lead to unpredictable behaviour.

Edit: Here's is a version of the brew function which leaves your conda environments as it found it:

brew() {
    local -a conda_envs
    while [ "$CONDA_SHLVL" -gt 0  ]; do
        conda_envs=("$CONDA_DEFAULT_ENV" $conda_envs)
        conda deactivate
    done
    command brew $@
    local brew_status=$?
    for env in $conda_envs; do
        conda activate "$env"
    done
    unset env
    return "$brew_status"
}

I've tested this in Zsh. I don't think it will work in Bash. If you want to use it in Bash, you will need to change the for loop declaration to say something like for env in ${conda_envs[@]}. I haven't tested this, however, so please test that it does what you need before use.

Solution 2

You can set up a virtual environment (virtualenv) and assign different versions Python to each so there wouldn't be any overlap causing dependency issues.

Check out 'The Hitchhiker's Guide to Python' on how to walk through the setup.

http://docs.python-guide.org/en/latest/dev/virtualenvs/

Solution 3

I am new to python, and have had trouble with my python installation. I have both python installation from homebrew and anaconda on my mac. My anaconda installation had messed up my homebrew python dependency for vim and MacVim.

My solution is whenever I am installing/update package via homebrew I would remove anaconda from my PATH. This is a bit of a pain, but I only installing/updating package via homebrew once in a while so it okay.

To switch between python 2 and 3 here are my commands in Terminal:

$ conda search python
...
...
$ conda install python=3.5.0

$ conda info -e
# conda environments:
#
py27                     /Users/apollotang/opt/local/anaconda2/envs/py27
py36                     /Users/apollotang/opt/local/anaconda2/envs/py36
root                  *  /Users/apollotang/opt/local/anaconda2

$ source activate py36      ### <——— here is the command in to change python version 
(py36) $ conda info -e
# conda environments:
#
py27                     /Users/apollotang/opt/local/anaconda2/envs/py27
py36                  *  /Users/apollotang/opt/local/anaconda2/envs/py36
root                     /Users/apollotang/opt/local/anaconda2

(py36) $ python -V
Python 3.6.2 :: Anaconda custom (x86_64)

I found this command from How to change default Anaconda python environment and http://chris35wills.github.io/conda_python_version/

Also, here is a very good article on best practice on setting up python environment on mac https://www.davidculley.com/installing-python-on-a-mac/

Solution 4

Installing anaconda would mean you have no need of the system Python (leave it to the system, since you probably don't want to change it), or of Pythons installed by homebrew.

When you create a conda environment you can specify which version of Python you want it to use. For example:

$ conda create --name myenv python=3.5

Activating a conda environment (after the above command you'd do this with conda activate myenv, or source activate myenv for earlier versions) automatically means that the python command will run the Python interpreter you selected for that environment. Packages can be installed with pip as well as conda install, and are pretty much compatible with conda environments.

Solution 5

If using .bash, the following will temporarily deactivate anaconda environment while running brew command(s). (see original zsh post)

Step 1

$ vim ~/.bash_profile

--add--

if [ -f ~/.bashrc ]; then
    source ~/.bashrc
fi

Step 2

$ vim ~/.bashrc

--add--

# Temporarily deactivates anaconda environment to run brew command(s)

brew() {(
    local conda_env="$CONDA_DEFAULT_ENV"
    while [ "$CONDA_SHLVL" -gt 0  ]; do
        conda deactivate
    done
    command brew $@
    local brew_status=$?
    return "$brew_status"
)}

View more solutions

Comments

[OneCricketeer]

Homebrew cask.... brew cask install anaconda

[shuhalo]

@cricket_007 That command freezes with 'PREFIX=/usr/local/anaconda3

[OneCricketeer]

@shuhalo Okay, well, I typically use pyenv instead of Homebrew for my Python

[cade]

@shuhalo it doesn't provide any updates for a while after that output, but it does not freeze on my machine.

[Mike Williamson]

I feel that @cricket_007 's appoach is the right approach, but: (a) it should be written as an answer, not just a comment, and (b) it should have more details, like why it is best to first install homebrew, then install conda via homebrew. I am not convinced this is the best way, even though I feel it is. The reasoning behind this method would be helpful, especially in terms of how it impacts your $PATH variable and where Anaconda and Python are stored.

[madcow]

Update: New homebrew syntax: brew install --cask anaconda

[Mark Ribau]

Would putting the entire body of the function inside a subshell (parenthesis pair) work for brew to function properly AND to "reactivate" (by exiting subshell) conda?

[Theoretical Economist]

@Mark Apologies, I haven’t been on SE in a while so I only just saw your comment. That may work, I’m not sure. Have you tested it out? One way to see is to try it with brew doctor, which will complain if it finds another version of Python in your PATH. As an aside, I think this function can be easily modified to keep track of your conda environment stack if you’re using Zsh.

[Theoretical Economist]

@Mark I've updated my answer with something you may be related to your question.

[Mark Ribau]

FYI, the parens method seems to work in bash, just make {->{(, }->)} and remove [ -n "${conda_env:+x}" ] && conda activate "$conda_env" from original