Disable HttpClient logging

Question

I´m using commons-httpclient 3.1 in an integration test suite. The default logging for HttpClient is extremely noisy and I can't seem to turn it off. I've tried following the instructions here but none of them make any difference.

Mostly I just need to make the org.apache.http.wire logger shut up. Part of the problem is that I don't know what type of logger HttpClient is trying to use. I've never used this library before. I tried creating a log4j.properties file and dropping it in my test/resources folder, modifying the master logging.properties file in jre/lib, and sending in the various logging options to Maven as specified on the logging page, and none of them make any difference.

UPDATE: A correction: it appears the output in question is actually originating through jwebunit's usage of HttpClient, not my own. Either way, it's not desirable.

UPDATE: Thanks for the attempts so far. I've tried everything suggested below but still no luck. I have a file commons-logging.properties in my src/test/resources folder with the following contents

org.apache.commons.logging.LogFactory=org.apache.commons.logging.impl.Log4jFactory
log4j.configuration=log4j.properties

and a file log4j.properties in the same folder with the following contents

log4j.rootLogger=ERROR, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%c] %m%n

#This is the line that should make httpclient shut up
log4j.logger.org.apache.http=ERROR

However, when I run my tests I still get a bunch of output like this:

21:57:41.413 [main] DEBUG org.apache.http.wire - << "                                   [\r][\n]"
21:57:41.413 [main] DEBUG org.apache.http.wire - << "[\r][\n]"
21:57:41.413 [main] DEBUG org.apache.http.wire - << "                                   [\r][\n]"
21:57:41.413 [main] DEBUG org.apache.http.wire - << "                               </ul>[\n]"
21:57:41.413 [main] DEBUG org.apache.http.wire - << "    [\n]"
21:57:41.424 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "[\r][\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "[\r][\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "                   </div>[\r][\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "                </li>[\r][\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "            [\r][\n]"
21:57:41.425 [main] DEBUG org.apache.http.wire - << "            [\r][\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "        </ul>[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.433 [main] DEBUG org.apache.http.wire - << "<div class="details">[\n]"
21:57:41.442 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "<div class="details-body details-precis  ">[\n]
"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "<div class="details-state">[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.443 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "</div>[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\n]"
21:57:41.455 [main] DEBUG org.apache.http.wire - << "[\r][\n]"
Destroying 1 processes21:57:41.465 [main] DEBUG org.apache.http.wire - << "[\r][\n]"

This output for everything that comes across the wire is making this library unusable for me...that is until I can figure out how to turn it off. Is there anything special I need to do to get this log configuration read in?

Solution 1

Update log4j.properties to include:

log4j.logger.httpclient.wire.header=WARN
log4j.logger.httpclient.wire.content=WARN

---

Note that if Log4j library is not installed, HttpClient (and therefore JWebUnit) will use logback. In this situation, create or edit logback.xml to include:

<configuration>
    <logger name="org.apache" level="WARN" />
    <logger name="httpclient" level="WARN" /> 
</configuration>

---

Setting the log level to WARN with Log4j using the package name org.apache.commons.httpclient in log4j.properties will not work as expected:

log4j.logger.org.apache.commons.httpclient=WARN

This is because the source for HttpClient (v3.1) uses the following log names:

public static Wire HEADER_WIRE = new Wire(LogFactory.getLog("httpclient.wire.header"));
public static Wire CONTENT_WIRE = new Wire(LogFactory.getLog("httpclient.wire.content"));

Solution 2

Note: Some of this answer might repeat things you already know (or think you know), but there is a bit of mis-information floating around on this question, so I'm going to start at the beginning and spell it all out

• Commons HttpClient uses Commons-Logging for all its logging needs.
• Commons-Logging is not a full logging framework, but rather, is a wrapper around several existing logging frameworks
• That means that when you want to control the logging output, you (mostly) end up configuring a library other than Commons-Logging, but because Commons-Logging wraps around several other libraries, it's hard for us to guess which one to configure without knowing your exactly setup.
• Commons-Logging can log to log4j, but it can also log to java.util.logging (JDK1.4 logging)
• Commons-Logging tries to be smart and guess which logging framework you are already using, and send its logs to that.
• If you don't already have a logging framework, and are running on a JRE that's 1.4 or above (which you really should be) then it will probably be sending its log messages to the JDK logging (java.util.logging)
• Relying on Commons-Logging's autodiscovery mechanism is prone to error. Simply adding log4j.jar onto the classpath would cause it to switch which logging mechanism it uses, which probably isn't what you want
• It is preferable for you to explicitly tell Commons-Logging which logging library to use
• You can do this by creating a commons-logging.properties file as per these instructions
• The steps you want to follow to configure the commons-httpclient logging are
  1. Decide which underlying logging framework you want to use. There are a number of choices, but probably log4j or java.util.logging are the best options for you.
  2. Set-up the commons-logging properties file to point to the correct Log implementation. e.g. to use log4j, put this into the properties file: org.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger, or to use JDK logging set org.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger. These can also be set as system properties (e.g. using -D on the command line).
  3. Configure the underlying logging implementation (e.g. log4j) to ignore the messages you don't want, and output the messages you do want.

That's a lot of steps, but that's what it takes. The developers at Apache-commons tend to assume you'll already have a logging framework configured, and they can work out which one it is by auto-discovery.
If that's not true for you, then it tends to be a bit more work to get things running.

Solution 3

For log4j, add the following to log4j.properties (in the application's source directory):

log4j.logger.org.apache=WARN
log4j.logger.httpclient=WARN

For logback, the following logback.xml will kill the noise:

<configuration>
    <logger name="org.apache" level="WARN" />
    <logger name="httpclient" level="WARN" /> 
</configuration>

Solution 4

This worked for my tests;

java.util.logging.Logger.getLogger("org.apache.http.wire").setLevel(java.util.logging.Level.FINEST);
java.util.logging.Logger.getLogger("org.apache.http.headers").setLevel(java.util.logging.Level.FINEST);
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime", "true");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire", "ERROR");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http", "ERROR");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http.headers", "ERROR");

Solution 5

I put this into my log4j config file

log4j.logger.org.apache.http.wire=WARN

This limits the output to Warning level or above

View more solutions

Comments

[Bob]

In the 4.2.1 source, the log names are: "org.apache.http.headers" and "org.apache.http.wire", though even using them, the noisy apache logging won't seem to shut off for me.

[T9b]

Thanks, I had the same problem when using OpenRdf. All the other tips in this thread seemed to have no effect until I removed the logback jars, now the log is nicely quiet.

[Jamel Toms]

This worked for me too, but I didn't need the: Logger.getLogger("org.apache.commons.httpclient").setLevel(L‌​evel.ERROR);

[FelixD]

Thank you!!! Also: If you have this issue somewhere in your own code, where you use httpclient and don't already use Log4j: Also remember to include the log4j jar in the classpath ...

[Arun Thundyill Saseendran]

I am using log4j. Adding the first two lines in the log solver my problem completely. All my other log messages appear according to the level I set. Whereas apache logs doesn't. Great help. Thanks.

[Manish Patel]

You beauty. I almost killed a cat because of this thing. It was driving me insane.

[Nathan]

This was the solution for me. I think its because other libraries I was including transitively included logback, so it got locked onto this logger.

[parsecer]

Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enfo‌​rce (enforce) on project gs-serving-web-content: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed

[Matt Baker]

It will likely vary depending on your situation, but for disabling the terribly verbose logging enabled out of the box with the AWS SDK, this is the only one that worked.

[lasote]

Fantastic, the only solution it worked for me. Thanks so much.

[ExactaBox]

Thank you! Was using the exact same code at the start of the test method, did nothing. Placing it in its own @Before or @BeforeClass function worked beautifully.

[Hikaru Shindo]

Where I need to put these sir ?

[irlatech]

@HikaruShindo this should go in your java code as part of initialization

[Anwar Reefat]

Only solution that worked for me. Initiated this inside WebApplicationInitializer.

[Julian Cardenas]

This is what you need if you're just running a local test from a main method :) thanks!

[Francislainy Campos]

Thank you. This fixed it for me. Had also to add <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-jcl</artifactId> <version>${log4j.logging.version}</version> </dependency>

[Xiaobo Gu]

Hi, how should we do with the xml config file for log4j, thanks.