How can I completely remove any logging from requests module in Python

Question

How can I completely remove any logging from requests module in Python? I don't need to set even CRITICAL level. Smth like this

import logging
requests_log = logging.getLogger("requests")
requests_log.setLevel(logging.CRITICAL)

but without any messages, even CRITICAL.

Solution 1

First of all, requests doesn't log anything; only the urllib3 library that requests depends on does. That library only logs messages at INFO and DEBUG levels, so setting the log level to logging.CRITICAL disables all messages already:

urllib3_log = logging.getLogger("urllib3")
urllib3_log.setLevel(logging.CRITICAL)

You could also just disable propagation:

logging.getLogger("urllib3").propagate = False

That's enough to completely disable all logging that the urllib3 library does.

The urllib3 project has installed a NullHandler() handler object on the project root logger, which ensures that the lastResort handler is not used for unhandled messages, even when propagation has been disabled.

That said, if you don't trust that future versions of requests won't use sys.maxint as a log level, and at the same time neglect to set a NullHandler(), then by all means add your own NullHandler() on the project root logger object, and then disable propagation there:

import logging

requests_log = logging.getLogger("requests")
requests_log.addHandler(logging.NullHandler())
requests_log.propagate = False

Now all logging within the requests hierarchy will be directed to the NullHandler() instance, and with propagate set to False logging stops there. You can use this to silence any logger in the hierarchy.

In my opinion, that's overkill, no requests release made so far uses logging, and the project is staffed with capable developers that are unlikely to misconfigure the logging setup if they ever did add logging.

Solution 2

The accepted answer didn't work out for me. But urllib3 docs point out that you can disable the logging with this:

import urllib3
urllib3.disable_warnings()

Source: urllib3 docs

Comments

[chanux]

I'm on python version 2.7.6 (requests 2.12.4, urllib3 1.7.1). Even after I do all these I get log entry per request. There's no indication where these logs are from though.

[Martijn Pieters]

@chanux: I'm sorry, why are you telling me your versions?

[chanux]

Sorry I pressed enter accidentally :). Any help is appreciated.

[Martijn Pieters]

@chanux: I updated the answer to be more clear that you'd need to disable logging for urllib3, not requests.

[Maggyero]

What is the purpose of the line requests_log.addHandler(logging.NullHandler())? The Logging HOWTO recommends to attach a logging.NullHandler to the top-level logger of your library, and only if you don't want your library's users to get your library's event messages of severity logging.WARNING or greater printed to sys.stderr in the absence of a logging configuration in their application. But here this is not the library code, this is the application code. So no need to redirect anything since propagate is False.

[Martijn Pieters]

@Maggyero: the point is that requests doesn't attach a NullHandler to its root logger (there isn't even a root logger). Placing a NullHandler there is only needed if you as an application developer don't trust a future version of requests to add logging and not take care of putting a NullHandler in place for you.

[Maggyero]

Thanks for the clarification: after looking again at the source code of the logging.Logger.callHandlers method, I realized that logging.getLogger("requests").propagate = False is not enough to prevent the fallback to the logggin.lastResort handler. So logging.getLogger("requests").addHandler(logging.NullHandler‌​) is necessary, but only if the application code doesn't define its own loggers. Can I add this to your answer?

[Martijn Pieters]

@Maggyero: I've expanded on the role of NullHandler, propagation and lastResort a little.

[Martijn Pieters]

@Maggyero: your edit shows you may be misunderstanding my answer entirely. I definitely meant to target requests, not urllib3, which already does the right thing as a library.

[Maggyero]

My bad, the main point of my edit (besides formatting improvement) was to precise that adding a logging.NullHandler() to the library logger is necessary only if the application has not attached a handler to the logging.RootLogger, since I thought in this case there would be no fallback to the logging.lastResort handler, as the found variable in logging.Logger.callHandlers would not be 0. But this is incorrect: propagate = False breaks the while loop, so the found variable does not get incremented by the handler attached to the logging.RootLogger parent logger, so it is 0.

[Maggyero]

But something is still confusing for the reader. You say: "The urllib3 project has installed a NullHandler() handler". Then later you say: "That said, if you don't trust that future versions of requests won't use sys.maxint as a log level, and at the same time neglect to set a NullHandler()"? In the 2nd sentence about requests, it looks like you are referring to the first sentence about urllib3, which is confusing. And to me you cannot trust any libraries, so the application code should add a NullHandler() to urllib3 too, to be safe (even if urllib3 currently has one).

[Martijn Pieters]

@Maggyero: no, I start with stating that requests doesn't log anything, only the urllib3 library. requests calls call urllib3 code, which log stuff. The OP silenced the requests library, but not urllib3. Now, a future update of requests could add logging itself, but forget to add a nullhandler (unlikely, but possible), then log using sys.maxint as the log level (far less likely still).

[variable]

General question: Within the package, how does adding null handler to child logger (name) prevent the root logger from displaying log messages (lastResort)? The log will still propagate to the root logger isnt it? There is always a root logger isn't it?

[Martijn Pieters]

@variable: That's not how propagation works, actually. The Logger class has a callHandlers method; this method doesn't recurse nor is it called on each logger object in the hierarchy. It is called once, on the logger object that you sent the log message to, and simply loops over the .parent chain to test each logger down to the root. This method will call lastResort, but only if no handlers are found anywhere.

[Martijn Pieters]

@variable: put differently: the root logger not having a handler doesn't mean it'll use lastResort, lastResort is really only used as a last resort when nothing else handled a log message, anywhere.

[variable]

My confusion is cleared when I realised that the root logger when running the library is different to the root logger when the library is imported.