How to resolve Python package dependencies with pipenv?

Question

I am using pipenv to handle Python package dependencies.

The Python package is using two packages (named pckg1 and pckg2) that rely on the same package named pckg3, but from two different versions. Showing the dependency tree :

$ pipenv graph
  pckg1==3.0.0
    - pckg3 [required: >=4.1.0]
  pckg2==1.0.2
    - pckg3 [required: ==4.0.11]

An attempt to install dependencies :

$ pipenv install

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches pckg3==4.0.11,==4.1.0,>=4.1.0 (from -r C:\Users\user\AppData\Local\Temp\pipenv-o7uxm080-requirements\pipenv-hwekv7dc-constraints.txt (line 2))
Tried: 3.3.1, 3.3.2, 3.3.3, 3.4.0, 3.4.2, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.0.3, 4.0.3, 4.0.4, 4.0.4, 4.0.6, 4.0.6, 4.0.8, 4.0.8, 4.0.9, 4.0.9, 4.0.10, 4.0.10, 4.0.11, 4.0.11, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.1.2, 4.1.2, 4.2.1, 4.2.1, 4.3.0, 4.3.0
There are incompatible versions in the resolved dependencies.

As suggested, pip install --skip-lock does the trick, but the dependency tree is still unresolved.

I would love to tell Pipenv to override pckg2's requirement, and specify pckg3>=4.1.0.

How can this be resolved?

Solution 1

I get that error constantly. Clearing the cache in the lock file works beautifully every time.

$ pipenv lock --pre --clear

Solution 2

You can't. At the moment, pipenv doesn't offer anything for an explicit override of requirement constraints.

As a workaround, you can put dependencies that you want to override to dev-packages as those will be overridden by packages, so this Pipfile should install pckg3>=4.1.0:

# Pipfile
...
[packages]
pckg1 = "==3.0.0"

[dev-packages]
pckg2 = "==1.0.2"

If you now lock and install:

$ pipenv lock --dev
$ pipenv install --dev

the requirement ==4.0.11 will be overridden by >=4.1.0. This is ugly if you ask me because this is not what development packages are meant for and you're changing the role of pckg2 dependency in project, but I don't see any better way here.

Solution 3

This works when there are unfinished routines on pipfile.

Once I made a mistake and run

pipenv install codecove # With an 'e' at the end

and the pipenv kept always trying to complete the installation with no success because the lib does not exist. I resolved it with:

pipenv uninstall codecove

and installed codecov after.

I tried to run

pipenv lock --clear
pipenv lock --pre --clear

but only after uninstalled the lib with wrong name I succeeded.

Solution 4

I have the similar issue with google-cloud-core.

$ pipenv lock
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0
There are incompatible versions in the resolved dependencies.

It was solved by

1. rm -rf Pipfile.lock
2. pipenv update

Solution 5

If you get an error like:

Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✘ Locking Failed! 

For me, this happened because the underlying virtual environment did not refer to my current directory.

I solved this problem by moving the contents to a new directory and deleting the old one.

I also had to delete the Pipfile and Pipfile.lock but I'm not sure if that's necessary.

View more solutions

Comments

[Val Berthe]

Overriding dependencies in dev-package works, but I really dislike putting non-dev packages in the dev packages list...

[hoefling]

I agree that stinks (mentioned that in the answer), but I'm afraid there's no other option available with pipenv's current development state...

[Marc]

This is helpful, but I had to do it without the --pre or I got other mismatch errors.

[Jake]

combining @northtree with Harsha Goli did the trick......but still have no real understanding of what the underlieing issue was :(

[Kermit]

This works for me... but I don't feel great about it. Ha.

[Connor]

@HashRocketSyntax yeah I agree. Wish I had transparency into the underlying problem

[arshbot]

It most often means there are invalid inconsistencies with the pipfile.lock. Sometimes this can genuinely happen if there is a dependency conflict - but this answer is tailored for when pipenv simply needs to dump it's cache and recreate the lock file

[ThePhi]

Yes it's very stupid! Even if I try to install pipenv install WrongName: Of course, it will fail to install but in addition it will make pipenv unable to install everything after that! So i need to do after that: pipenv uninstall WrongName...

[abhiieor]

@hoefling I don't understand how 3.0.0 and 1.0.2 is yaml are related to ==4.0.11 and >=4.1.0 in your explanation

[hoefling]

@abhiieor check out the output of pipenv graph in the question - the first two are versions of direct dependencies, the last two are conflicting versions of the same transient dependency. Overall, this answer may be obsolete, I don't use pipenv anymore.

[Ali Husham]

on my mac I can not use pipenv lock --pre --clear but it work only when i say python3.9 -m pipenv lock --pre --clear any help?

[Chidiebere]

In my own situation I get the dependency error when I run pipenv lock --pre --clear.

[Chidiebere]

``` ERROR: Could not find a version that matches werkzeug<2.0.0,>=0.7,>=2.0 (from -r /var/folders/2y/061kgpfx29110d440z4w1ft40000gn/T/pipenvr4na4‌​5z1requirements/pipe‌​nv-ez8fnckw-constrai‌​nts.txt (line 28)) Tried: 0.1, 0.2, 0.3, 0.3.1, 0.4, 0.4.1, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 0.9, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.10, 0.10.1, 0.10.2, 0.10.2, 0.10.4, 0.10.4, 0.11, 0.11, 0.11.1, 0.11.1, 0.11.2, 0.11.2, 0.11.3, 0.11.3, 0.11.4, 0.11.4, 0.11.5, 0.11.5, 0.11.6, ... ```