pip packages not included in "conda env export"

15,375

Solution 1

I was able to solve this problem just by running this in a command window with admin privileges:

activate <env-name>
conda update pip

After that, exports of the available packages on my environment contain again both the packages installed with conda and those with pip.

I suspect I may have caused this package export problem because I had followed the instruction that pip gave earlier on, when I was executing a package install:

"You are using pip version 9.0.3, however version 10.0.1 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' > command."

So updating with python -m pip install --upgrade pip broke my package exports, but conda update pip fixed it.

Solution 2

For me this only appeared using the conda package pip 18.1_py36_0 (didnt try any other versions). So i downgraded pip in the virtual environment to version 10 via

conda install pip=10

After this the export was listing the pip packages again.

Also see this pull request for conda https://github.com/conda/conda/pull/7612 which fixes the issue for conda 4.6.2. It also was backported to conda 4.5.10. Currently there is no conda 4.4.x backport.

Solution 3

At the moment of writing this post, because of an issue with reading pip lists by anaconda, conda export command only exports packages installed via conda install packagename and ignores the pip ones.

You can try reinstalling your packages via conda install and see if it generates the environment.yml file correctly.

Share:
15,375

Related videos on Youtube

Sander Vanden Hautte
Author by

Sander Vanden Hautte

Updated on June 23, 2022

Comments

  • Sander Vanden Hautte
    Sander Vanden Hautte almost 2 years

    My python project runs on a conda virtual environment. I install packages on the environment with conda install whenever that package is available for installing that way, otherwise, I install it with pip install.

    To make the project installation easier for other developers, I export the list of packages that are used on the conda environment to a file: conda env export > conda_environment_export.yml.

    However, since a while, the package export does no longer contain the packages installed by pip.

    What can I do to make pip packages appear again in the export?

    My conda export file as it was some weeks ago:

    name: sensor_gateway
    channels:
      - defaults
    dependencies:
      - aiohttp=2.3.9=py36_0
      - async-timeout=2.0.0=py36hc3e01a3_0
      - certifi=2018.1.18=py36_0
      - chardet=3.0.4=py36h420ce6e_1
      - jinja2=2.10=py36h292fed1_0
      - markupsafe=1.0=py36h0e26971_1
      - multidict=3.3.2=py36h72bac45_0
      - pip=9.0.1=py36h226ae91_4
      - pymysql=0.7.11=py36hf59f3ba_0
      - python=3.6.4=h6538335_1
      - pytz=2018.3=py36_0
      - setuptools=38.4.0=py36_0
      - sqlalchemy=1.2.1=py36hfa6e2cd_0
      - vc=14=h0510ff6_3
      - vs2015_runtime=14.0.25123=3
      - wheel=0.30.0=py36h6c3ec14_1
      - wincertstore=0.2=py36h7fe50ca_0
      - yarl=0.14.2=py36h27d1bf2_0
      - pip:
        - aiohttp-jinja2==0.16.0
        - aiomysql==0.0.12
        - attrs==17.4.0
        - idna==2.6
        - idna-ssl==1.0.0
        - python-mimeparse==1.6.0
    prefix: C:\ProgramData\Anaconda3\envs\sensor_gateway
    

    The conda export now:

    name: sensor_gateway
    channels:
      - defaults
    dependencies:
      - aiohttp=2.3.9=py36_0
      - async-timeout=2.0.0=py36hc3e01a3_0
      - certifi=2018.4.16=py36_0
      - chardet=3.0.4=py36h420ce6e_1
      - icc_rt=2017.0.4=h97af966_0
      - intel-openmp=2018.0.0=8
      - jinja2=2.10=py36h292fed1_0
      - markupsafe=1.0=py36h0e26971_1
      - mkl=2018.0.2=1
      - mkl_fft=1.0.1=py36h452e1ab_0
      - mkl_random=1.0.1=py36h9258bd6_0
      - multidict=3.3.2=py36h72bac45_0
      - numpy=1.14.2=py36h5c71026_1
      - pip=9.0.1=py36h226ae91_4
      - pymysql=0.7.11=py36hf59f3ba_0
      - python=3.6.4=h6538335_1
      - pytz=2018.3=py36_0
      - setuptools=38.4.0=py36_0
      - simplejson=3.14.0=py36hfa6e2cd_0
      - sqlalchemy=1.2.1=py36hfa6e2cd_0
      - vc=14=h0510ff6_3
      - vs2015_runtime=14.0.25123=3
      - wheel=0.30.0=py36h6c3ec14_1
      - wincertstore=0.2=py36h7fe50ca_0
      - yarl=0.14.2=py36h27d1bf2_0
    prefix: C:\ProgramData\Anaconda3\envs\sensor_gateway
    

    I suspect that updating the version of conda (4.4.10-py36_0 --> 4.5.2-py36_0) may have caused this problem on this project. On another project, I have the same issue, but there I updated pip (9.0.1-py36h226ae91_4 --> 10.0.1-py36_0). Even after going back to pip 9.0.1 there, the problem remains. Is my suspicion correct? And what can I do to fix this again?

    UPDATE:

    The only proposed answer right now links to this question and it does not seem to solve my problem:

    1. On a Windows command terminal, opened as administrator:

    where pip output (where as windows-equivalent command to which on unix/linux):

    C:\ProgramData\Anaconda3\Scripts\pip.exe
    C:\Users\Sander\AppData\Local\Programs\Python\Python36\Scripts\pip.exe
    

    pip freeze output:

    Lists all packages I ever installed on my 'normal' python (= everything by default installed, not inside any dedicated virtual environments). The package list is too long to include here.

    2. On a Windows command terminal, opened with administrator privileges, and after executing activate sensor_gateway:

    where pip output:

    C:\ProgramData\Anaconda3\envs\sensor_gateway\Scripts\pip.exe
    C:\ProgramData\Anaconda3\Scripts\pip.exe
    C:\Users\Sander\AppData\Local\Programs\Python\Python36\Scripts\pip.exe
    

    pip freeze output:

    lists all packages installed with pip on my environment correctly.

    So it seems that when my virtual environment is activated, there is a pip installed in that environment, and it must be that pip that returns the list of installed packages when asked for, not another pip install, since the pip package list on the environment is correct.

    • Admin
      Admin over 5 years
      I wanna add that I had the same problem and it is solved by updating conda using conda update -n base -c defaults conda.
    • Stephen
      Stephen about 3 years
      Just wanted to note that if you use --from-history then pip packages will not be included. Some people in that situation may find this post. What you want to do is follow this GitHub issue where it is being discussed: github.com/conda/conda/issues/9628
  • Sander Vanden Hautte
    Sander Vanden Hautte almost 6 years
    Sorry, but this can't be the solution. This is how I make the export of my environment right now, and I had pip installed with conda. The conda-env export on my 2 environments indeed included pip packages automatically earlier on, but not anymore.
  • Sander Vanden Hautte
    Sander Vanden Hautte almost 6 years
    Your link to a related question was interesting though. I added some info in my question as to why I think my problem is not related to a call to a wrong pip install, as suggested in that related question.
  • Avi
    Avi about 5 years
    I downgraded my conda environment to pip=9.0.1 similarly and solved the issue.