OS X - Deciding between anaconda and homebrew Python environments

Question

I use Python extensively on my Mac OS X, for both numerical applications and web development (roughly equally). I checked the number of Python installations I had on my laptop recently, and was shocked to find four:

Came with Mac OS X:
/usr/bin/python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin

Installed via Homebrew
/usr/local/bin/python
Python 2.7.10 (default, Jul 13 2015, 12:05:58)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin

Installed via Anaconda/Miniconda
~/anaconda/bin/python
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Oct 19 2015, 18:31:17)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org

Came with the downloaded .pkg from python.org
/System/Library/Frameworks/Python.framework/Versions/Current/bin/python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin

I decided to unify all of this, and use conda. I removed the Homebrew version and the Python.org download (kept the main system one). Conda is great for numerical computing, because I can install Jupyter/Numpy/Pandas in the root environment, and not have to bother install virtualenvs for every project.

But now my entire web development workflow is messed up. None of my virtualenvs work, since apparently one's not supposed to use conda and virtualenv together. I tried to create conda environments from the requirements.txt file. One package I was using with django was "markdown_deux", which is not available in the Conda repo. I looked at ways of building it, but creating a recipe takes a lot of effort (create YAML file, etc..)

Has anyone found a good compromise for this? I'm thinking of going back to the homebrew version for general use, and writing an alias for changing the path back to the conda version as necessary. Though this will also require tracking which one I'm using now..

Solution 1

I use Homebrew Python for all my projects (data science, some web dev).

Conda is nothing fancy, you can have the same packages by hand with a combination of pip and Homebrew science. Actually, it is even better because you have more control on what you install.

You can use your virtualenvs only when you do web development. For the numerical applications you will probably want to have the latest versions of your packages at all times.

If you want to update all your packages at once with pip, you can use this command:

sudo -H pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 sudo -H pip install -U

EDIT: This answer is old, if you want a more up-to-date comparison, I found this nice blog article which compares the two approaches:

https://towardsdatascience.com/pipenv-vs-conda-for-data-scientists-b9a372faf9d9

I still use Homebrew Python, and pip / pipenv over conda.

Solution 2

Workflow that I've found the best:

• Use conda for virtual environment management. Never use / install into system python.

• Use pip to install into the active virtual environment, just like normal.

• Use conda packages only for hard to install software, such as Qt.

Automation / extras

• Use autoenv or direnv and automatically activate virtual environments when you enter a directory by putting the conda command inside the .env or .envsrc file.