Hudson build successful with unit test failures

Question

This is related to another question I asked about trying to fail Hudson on a single unit test failure.

When I run Maven on my build standalone, it fails due to unit test errors (running mvn clean install) but when I run it in Hudson, it says there was an error due to test failure, yet the build still is successful (and marked as unstable). Any thoughts?

Solution 1

What type of Hudson job did you create? I suspect it's the m2 project rather than the freestyle project. In the m2 project, Hudson will actually tweak the way your pom is configured so as to prevent failures and instead mark it as unstable. This was intentional.

Try setting maven.test.failure.ignore=false under the Advanced setting as I believe Hudson automatically sets that to true.

Solution 2

Hudson's convention is that it will only look for tests if the build returned success. If the build failed hudson interprets that to mean there was some unrecoverable error that caused the build to not work, and not that there were some tests that failed.

It's quite useful differential as thus unstable (yellow) means the build completed and the tests were all able to run, but some failed.

And error means that the build failed in that it couldn't compile or that there was some other unrecoverable error that caused the tests not to run, in this case hudson won't report on the unit test state.

Comments

[bendin]

ha! I just discovered this behavior today. One wrinkle: Hudson will successfully build my project with failing unit tests, but silently fails to build projects which depend on it. (I've selected is for downstream projects to be rebuilt automatically when snapshot dependencies are built. In this case, though, this doesn't occur. That was cause for considerable head scratching today.)

[palacsint]

The link is broken. Send me a comment please if you fixed it.