How do I disable log messages from the Requests library?

Question

By default, the Requests python library writes log messages to the console, along the lines of:

Starting new HTTP connection (1): example.com
http://example.com:80 "GET / HTTP/1.1" 200 606

I'm usually not interested in these messages, and would like to disable them. What would be the best way to silence those messages or decrease Requests' verbosity?

Solution 1

I found out how to configure requests's logging level, it's done via the standard logging module. I decided to configure it to not log messages unless they are at least warnings:

import logging

logging.getLogger("requests").setLevel(logging.WARNING)

If you wish to apply this setting for the urllib3 library (typically used by requests) too, add the following:

logging.getLogger("urllib3").setLevel(logging.WARNING)

Solution 2

In case you came here looking for a way to modify logging of any (possibly deeply nested) module, use logging.Logger.manager.loggerDict to get a dictionary of all of the logger objects. The returned names can then be used as the argument to logging.getLogger:

import requests
import logging
for key in logging.Logger.manager.loggerDict:
    print(key)
# requests.packages.urllib3.connectionpool
# requests.packages.urllib3.util
# requests.packages
# requests.packages.urllib3
# requests.packages.urllib3.util.retry
# PYREADLINE
# requests
# requests.packages.urllib3.poolmanager

logging.getLogger('requests').setLevel(logging.CRITICAL)
# Could also use the dictionary directly:
# logging.Logger.manager.loggerDict['requests'].setLevel(logging.CRITICAL)

Per user136036 in a comment, be aware that this method only shows you the loggers that exist at the time you run the above snippet. If, for example, a module creates a new logger when you instantiate a class, then you must put this snippet after creating the class in order to print its name.

Solution 3

import logging
urllib3_logger = logging.getLogger('urllib3')
urllib3_logger.setLevel(logging.CRITICAL)

In this way all the messages of level=INFO from urllib3 won't be present in the logfile.

So you can continue to use the level=INFO for your log messages...just modify this for the library you are using.

Solution 4

For anybody using logging.config.dictConfig you can alter the requests library log level in the dictionary like this:

'loggers': {
    '': {
        'handlers': ['file'],
        'level': level,
        'propagate': False
    },
    'requests.packages.urllib3': {
        'handlers': ['file'],
        'level': logging.WARNING
    }
}

Solution 5

import logging

# Only show warnings
logging.getLogger("urllib3").setLevel(logging.WARNING)

# Disable all child loggers of urllib3, e.g. urllib3.connectionpool
logging.getLogger("urllib3").propagate = False

View more solutions

Comments

[aknuds1]

What's the point of being more specific than just 'requests' though, from a practical POV?

[aknuds1]

But what do you gain by calling logging.getLogger("requests.packages.urllib3") instead of logging.getLogger("requests"), considering that you want to affect the logging of the requests library?

[aknuds1]

Do you mean that you wish to enable logging within requests.packages.urllib3? If so, you're answering the wrong question.

[sorin]

@aknuds1 It's up to you if you want to disable or enable them, I only put the code that fully controls this :)

[aknuds1]

I think you've misunderstood the question's scope.

[Janith Chinthana]

I have same issue with pysimplesoap, and this answer help me save my day

[jpoppe]

You could combine the two lines like this: logging.getLogger('requests').setLevel(logging.WARNING)

[aknuds1]

@JasperPoppe Thanks for the suggestion, implemented.

[dgassaway]

I had to add this line for the "urllib3" logger to suppress requests log messages.

[m_messiah]

I was need to import logging; logging.getLogger("urllib3").setLevel(logging.WARNING), too. Logger for "requests" doesn't prevent these messages.

[robru]

For some reason when using the requests library in python3 you have to do getLogger("urllib3") to suppress the messages.

[aknuds1]

@Robru Sounds as if requests is basing itself on urllib3 these days quite simply, which does logging of its own. That's how it goes :)

[razz0]

I suggest using setLevel(logging.WARNING) to log also possible warning and error messages.

[tripleee]

I did not find this method in my version. Disabling warnings is excessive, as the annoying messages are level INFO.

[Frank Meulenaar]

Is there a way to make change the level of the logging from requests to DEBUG instead of INFO? I would like to keep the messages, but only at the DEBUG level (so they are only in my offline debug log, and not on the screen).

[aknuds1]

@FrankMeulenaar I would pose that as an independent question. Anyhow, I guess what I would do is configure the requests stream handler outputting to screen to ignore anything below or equal to INFO.

[Bob Dem]

Thank you, this helped me silence urllib3 log messages when using boto3. The logger in such case is botocore.vendored.requests.packages.urllib3, so I used this: logging.getLogger("botocore.vendored.requests.packages.urlli‌​b3").setLevel(loggin‌​g.WARNING) and I finally got rid of the messages.

[Daniel García Baena]

I set it to CRITICAL but I'm still getting the same log message: 2017-08-05 15:17:13,334 ERROR -- : Certificate did not match expected hostname: www.improving-autonomy.org. Certificate: {'subjectAltName': [('DNS', '*.wordpress.com'), ('DNS', 'wordpress.com')], 'subject': ((('commonName', u'*.wordpress.com'),),)} Any help will be highly appreciated

[Daniel García Baena]

Solved. Full path was needed: logging.getLogger("requests.packages.urllib3").setLevel(logg‌​ing.CRITICAL)

[Robert Townley]

Many thanks for this! Altering the print criteria allowed me to isolate that python-elasticsearch was the culprit in my case.

[user136036]

Be aware that this will not work when modules create their loggers inside their class you'd call later, like the APScheduler does when you call BackgroundScheduler.BackgroundScheduler().

[Henri-Maxime Ducoulombier]

@JanithChinthana Your comment saved my day also

[Patrick]

This is a perfectly valid answer for people who use configuration file. Not sure why it got so many down vote?

[uhbif19]

@SebastianWagner Django uses dictConfig under the hood.

[Martijn Pieters]

Current versions of the requests library do not vendor urllib3 any more and so doesn't log anything. You only need to ask the urllib3 library to not propagate: logging.getLogger("urllib3").propagate = False, or increase the logging level if you still need to see warnings or critical messages (only warnings are issued currently).

[Martijn Pieters]

@user136036: logger objects are singletons, it doesn't matter if you or the library get to create them first. If you use the exact same name as the library uses, it will work.

[kbrose]

I think they are saying that if you list the loggers before a library has created its logger, then it won’t be listed. Which is correct.

[MehmedB]

Thank you so much! This is pretty good. One place to rule all of the library logs!! :)

[WestCoastProjects]

where are these levels set?

[Mikko]

I have them in Django settings, in base.py. Where to put them of course depends on your project setup.

[stinodego]

This helped me find the module that was writing log messages with request information that I wanted to get rid off (azure). Thank you so much!