Generating JUnit reports from the command line

Question

I have a test setup for a cloud system that uses a mixture of python for process level control and junit for internal state inspection. Essentially, I bring up several VMs to server as the cloud and then a junit VM which is a member of the cloud but drives tests and checks internal state. Our existing cloud management stuff is driven by python and I would like to maintain this.

I have a working setup that will run the JUnit command line via

java -ea -cp <classpath> org.junit.runner.JUnitCore <tests>

but this does not produce an report file. I know that ant is capable of producing an xml report, but I do not want to involve ant in this process (I have enough moving parts already).

Is there a way to launch junit from the command line such that it produces a report?

Ideally, I would have the junit tests produce xml reports, the python tests produce xml reports, and then merge them together for consumption by our CI system.

Update: The command line execution must support Windows, Linux, and Mac. We are not allowed to ship an external ant, although packaging an internal ant might be an option.

Solution

The JUnit library does not have any XML output options. To achieve such a thing, you'll need to write your own RunListener, which listens for the output and will in your case write the XML file.

However, to get the XML file in the correct format so that it can be read by CI system, I think it would be far easier to just use ant, either via the command line using a build.xml (JUnitReport), or using the java api: How can i use Apache ANT Programmatically.

EDIT: Initially, we had four options:

1. Use ant from the command line
2. Use ant programmatically (using the Java API)
3. Use the XMLJUnitResultFormatter directly with JUnitCore
4. Create a custom RunListener which produces the correct XML output.

Given the restrictions added by the OP, we can't use ant from the command line, which eliminates 1.

After looking more closely at the Ant JUnit task, it seems to be impossible to use this with JUnitCore (adding a TestListener), because ant uses the name of the test class directly, so you can't do a bridge class. From XMLJUnitResultFormatter.java

private void formatError(String type, Test test, Throwable t) {
    ...
    nested.setAttribute(ATTR_TYPE, t.getClass().getName());

    String strace = JUnitTestRunner.getFilteredTrace(t);
    Text trace = doc.createTextNode(strace);
    nested.appendChild(trace);
}

This eliminates 3.

Invoke Ant programmatically, via the Java API. I can't find any recent documentation on this. This seems to be hard.

So, finally, I would do 4, a custom RunListener, using the code from XMLJUnitResultFormatter as a base. And then, I'd publish it on github.com, so this question could be answered properly :-)

Comments

[Matt]

Sorry, I seem unable to make linebreaks appear... If I understand you correctly, you are suggesting that I should: 1) include the ant jar 2) write a custom main 3) add the ant jar's RunListener to a JUnitCore instance 4) let it rip? or is there a shorter version of the form 1) include the ant jar 2) invoke it more directly and it will handle the JUnitCore portion?

[Matthew Farwell]

Ant invokes junit, it runs the tests. So you'd provide it with a build.xml, and invoke ant. The simplest way would be to ensure that ant was installed somewhere on your system, and then just do it from the command line. Instead of java... you'd have ant...

[Matt]

Alas, requiring an ant install is one of the things I cannot do. This message came down from on high.

[Matt]

Thanks, I was afraid the answer would look like that, but definitely helpful.

[Barry Pitman]

I found such a RunListener in one of the downloads of Schmant, AntXmlRunListener. I extracted the required classes to github.com/barrypitman/JUnitXmlFormatter for easy reference.

[Timmos]

@BarryPitman Great man, I spent quite some time looking for XML generating without dependencies on ANT or so. It's very hard to find. Remarkable that your project is so hard to find.