Checkstyle vs. PMD

Question

We are introducing static analysis tools into the build system for our Java product. We are using Maven2 so Checkstyle and PMD integration come for free. However it looks like there is a large overlap in functionality between these two tools, in terms of enforcing basic style rules.

Is there a benefit from utilizing both of these? I don't want to maintain 2 tools if one will work. If we choose one, which one should we use and why?

We are also planning on using FindBugs. Are there other static analysis tools we should look at?

Update: Consensus seems to be that PMD is preferred over CheckStyle. I don't see a solid reason to use both, and I don't want to maintain 2 sets of rule files, so we will probably aim for PMD exclusively. We'll also be bringing in FindBugs, and perhaps, eventually, Macker to enforce architectural rules.

Solution 1

You should definitely use FindBugs. In my experience, the false-positive rate is very low, and even the least-critical warnings it reports are worth addressing to some extent.

As for Checkstyle vs. PMD, I would not use Checkstyle since it is pretty much only concerned with style. In my experience, Checkstyle will report on a ton of things that are completely irrelevant. PMD on the other hand is also able to point out questionable coding practices and its output is generally more relevant and useful.

Solution 2

If we choose one, which one should we use and why?

These tools are not competing but are complementary and should be used simultaneously.

The convention type (Checkstyle) is the glue that enables people to work together and to free up their creativity instead of spending time and energy at understanding inconsistent code.

Checkstyle examples:

• Is there javadoc on public methods ?
• Is the project following Sun naming conventions ?
• Is the code written with a consistent format ?

while PMD reminds you bad practices:

• Catching an exception without doing anything
• Having dead code
• Too many complex methods
• Direct use of implementations instead of interfaces
• Implementing the hashcode() method without the not equals(Object object) method

source: http://www.sonarsource.org/what-makes-checkstyle-pmd-findbugs-and-macker-complementary/

Solution 3

We use both:

• Checkstyle to make sure that everyone in the team write code in a similar maner
• PMD to find problematic code areas and next refactoring targets

Solution 4

If your primary place of use is while developing in eclipse, then CodePro from Instantiations will be best. Earlier it was a commercial tool, but now Google bought Instantiations so CodePro analytix is free now.

Check out http://code.google.com/javadevtools/download-codepro.html

Solution 5

If you reviewed Checkstyle, PMD and Findbugs rule lists, you have seen that all three provide valuable output and all three overlap to a degree and also bring their own, unique rules to the table. This is why tools like Sonar use all three.

That said, Findbugs has the most specific or niche rules (e.g. "Dubious catching of IllegalMonitorStateException" - how often are you likely to run into that?) so it is usable with little or no configuration and its warnings should be taken seriously. With Checkstyle and PMD the rules are more general and style-related so they should only be used with custom configuration files to spare the team from an avalanche of irrelevant feedback ("Tab char on line 5", "Tab char on line 6", "Tab char on line 7"... you get the picture). They also provide powerful tools to write your own advanced rules, e.g. the Checkstyle DescendentToken rule.

When using all three (especially with a tool like Sonar), all of them should be configured separately (takes at least a few days to cover all the rules) while paying attention to prevent duplication (all three tools detect that hashCode() has been overridden and equals() not, for example).

In summary, if you consider static code analysis valuable, rejecting the value any of the three provides makes no sense, but to use all three, you have to invest time to configure them to give you usable feedback.

View more solutions

Comments

[Jigar Shah]

One good thing about checkstyle is it allows some flexible rule like RegexpSingleline...

[John Tobler]

+1 for adding your recommendation of FindBugs. However, I strongly disagree with your opinion on Checkstyle unless you are a lone-wolf developer with your own idiosyncratic style. For teams, agreeing upon a common reasonable subset of rules and then using an automated tool like Checkstyle to automatically enforce them results in code that is readable by all.

[DPM]

While not perfect, FindBugs is the best by far. PMD and checkstyle point you towards downright bad practices. To be avoided at all cost unless you know very well which warnings are valid and which aren't.

[Roman Ivanov]

I agree that Checkstyle, more focus on code format and forcing developer to follow "Code standard", but it also could detect a lot of bad practices, take a look here, and extension of Checkstyle is more easy for development, but I do agree that it have limitations and will never overcome PMD and FindBug.

[barfuin]

True in 2008, but today Checkstyle has picked up a lot of speed.

[Adam Arold]

How do I configure it to get a report (in some usable format)? Now it only spits it to the console even if I configure log4j. I see that there is a bug report which might be related, but I'm not sure.

[Adam Arold]

We are thinking the same but in order to fix it I need it to be highlighted in my code or something similar. At least your settings.jar helped.

[xenoterracide]

Oddly enough I've had the exact opposite experience with PMD vs Checkstyle. PMD often reports false positives if it's something checkstyle or Findbugs didn't find. 7 years can matter a lot though.

[Christophe Roussy]

There are also ways to integrate it in the build and to make it fail on violation (with maven for example). I have done this for Checkstyle, PMD and FindBugs. As you said reporting is not enough.

[skomisa]

FYI, SpotBugs is the "spiritual successor of FindBugs", and the SpotBugs documentation is pretty good. As far as I know FindBugs hasn't been updated for years.

[Christophe Roussy]

Never heard of SpotBugs, probably because FindBugs + fbcontrib was enough for a long time, good to know there is some replacement

[Christophe Roussy]

There is some discussion about it here: news.ycombinator.com/item?id=12885549

[ThrawnCA]

It's also worth noting that tools tend to have configurable sensitivity. Eg when starting out with FindBugs/SpotBugs you might need to choose a High threshold to catch only the most serious bugs, then lower the threshold as you get things fixed.

[Christophe Roussy]

@ThrawnCA yes but even with sensitivity: on a large project too many errors will be found to be fixed in a reasonable amount of time. So what I do instead is add one rule at a time, starting with lowest hanging fruits like potential NP detection, then move on to rules like unclosed resources.