VSCode pytest test discovery fails

Question

Pytest test discovery is failing. The UI states:

Test discovery error, please check the configuration settings for the tests

The output window states:

Test Discovery failed: 
Error: Traceback (most recent call last):
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\run_adapter.py", line 16, in <module>
    main(tool, cmd, subargs, toolargs)
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\pytest.py", line 43, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 3)

Here are my settings:

{
    "python.pythonPath": ".venv\\Scripts\\python.exe",
    "python.testing.pyTestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pyTestEnabled": true
}

I can run pytest from the command line successfully FWIW.

Solution 1

I resolved the issue by upgrading pytest to the latest version: 4.4.1 with "pip install --upgrade pytest". I was apparently running an old version 3.4.2

Solution 2

I spent ages trying to decipher this unhelpful error after creating a test that had import errors. Verify that your test suite can actually be executed before doing any deeper troubleshooting.

pytest --collect-only is your friend.

Solution 3

This is not a complete answer as I do not know why this is happening and may not relate to your problem, depending how you have your tests structured.

I resolved this issue by putting an __init__.py file in my tests folder

E.G.:

├───.vscode
│       settings.json
│
├───app
│       myapp.py
│
└───tests
        test_myapp.py
        __init__.py

this was working a few days ago without this but the python extension was recently updated. I am not sure if this is the intended behavior or a side effect of how discoveries are now being made

https://github.com/Microsoft/vscode-python/blob/master/CHANGELOG.md
Use Python code for discovery of tests when using pytest. (#4795)

Solution 4

I just thought I would add my answer here as this might well affect someone who uses a .env file for their project's environment settings since it is such a common configuration for 12 factor apps.

My example assumes that you're using pipenv for your virtual environment management and that you have a .env file at the project's root directory.

My vscode workspace settings json file looks like below. The crucial line for me here was "python.envFile": "${workspaceFolder}/.env",

{
    "python.pythonPath": ".venv/bin/python",
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true,
    "python.linting.pycodestyleEnabled": false,
    "python.linting.flake8Enabled": false,
    "python.linting.pylintPath": ".venv/bin/pylint",
    "python.linting.pylintArgs": [
        "--load-plugins=pylint_django",
    ],
    "python.formatting.provider": "black",
    "python.formatting.blackArgs": [
        "--line-length",
        "100"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestPath": ".venv/bin/pytest",
    "python.envFile": "${workspaceFolder}/.env",
    "python.testing.pytestArgs": [
        "--no-cov"
    ],
}

I hope this saves someone the time I spent figuring this out.

Solution 5

In my case the problem with vscode being unable to discover tests was the coverage module being enabled in the setup.cfg (alternatively this could be a pytest.ini), i.e.

addopts= --cov <path> -ra

which caused the test discovery to fail due to low coverage. The solution was to remove that line from the config file.

Also, as suggested in the documentation:

You can also configure testing manually by setting one and only one of the following settings to true: python.testing.unittestEnabled, python.testing.pytestEnabled, and python.testing.nosetestsEnabled.

EDIT:

Slightly related - in case of more complex projects it might be also necessary to change the rootdir parameter (inside your settings.json) when running tests with pytest (in case of ModuleNotFoundError):

    "python.testing.pytestArgs": [
        "--rootdir","${workspaceFolder}/<path-to-directory-with-tests>"
    ],

View more solutions