pytest - specify log level from the CLI command running the tests

Question

my team and I are using Pytest + Jenkins to automate our product testing. we have been using the standard Logging lib of python to get proper log messages during testing, before and after each test etc. we have multiple layers of logging, we log out ERROR, WARNING, INFO and DEBUG. the default value for our logger is INFO. we create the logger object in the primary setup of the tests, and pass it down to each object created, so all our logs go to the same logger.

so far when we are developing a new feature or test, we are working in DEBUG mode locally, and change it back to INFO when submitting new code to our SVN, but i am trying to add option to change logging level using the CLI, but i haven't found anything easy to implement. I've considered using Fixtures, but from what i understand those are only for the tests themselves, and not for the setup/tear-down phases, and the log is created regard less of the tests. any hack or idea on how to add a Pytest option to the CLI command to support changing logging level?

Solution 1

Pytest 3.5 introduced a parameter

pytest --log-cli-level=INFO

For versions prior to 3.5 you can combine pytest commandline options and the python loglevel from commandline example to do the following:

Add the following to conftest.py:

import pytest
import logging


def pytest_addoption(parser):
    parser.addoption(
        "--log", action="store", default="WARNING", help="set logging level"
    )


@pytest.fixture
def logger():
    loglevel = pytest.config.getoption("--log")
    logger = logging.getLogger(__name__)

    numeric_level = getattr(
        logging,
        loglevel.upper(),
        None
    )
    if not isinstance(numeric_level, int):
        raise ValueError('Invalid log level: %s' % loglevel)

    logger.setLevel(numeric_level)
    return logger

and then request the logger fixture in your tests

def test_bla(logger):
    assert True
    logger.info("True is True")

Then run pytest like

py.test --log INFO

to set the log level to INFO.

Solution 2

Try --log-cli-level=INFO

like:

pytest -vv -s --log-cli-level=INFO --log-cli-format="%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)" --log-cli-date-format="%Y-%m-%d %H:%M:%S" ./test_file.py

Solution 3

This is now built into pytest. Just add '--log-level=' to the command line when running your test. For example:

pytest --log-level=INFO

Documentation updates can be found here: https://docs.pytest.org/en/latest/logging.html

Comments

[Pat Myron]

docs.pytest.org/en/latest/how-to/logging.html

[Nils Werner]

Does this answer your question?

[Avishay Cohen]

not exactly, but your example did give me the right idea. i'm creating the logger object at the setup_class of the suite, so i didn't use fixtures, i just used the example you gave me of how to use the pytest_addoption method, to add logging level to the pytest CLI command. i didn't realize it was that simple, and got lost in fixtures and hooks when reading in the pytest doc.

[Dennis Golomazov]

Thanks, this worked better than --log-cli-level, disabling third-party modules logging.

[Eray Erdin]

Oddly, this did not work for me, both my library and 3rd party.