Surefire is not picking up Junit 4 tests

Question

For some reason I cannot get Maven 2 Surefire plugin to execute JUnit 4 test class.

public class SimpleTest {
  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

However if I change this class to be JUnit-3 like, such as

public class SimpleTest extends junit.framework.TestCase {
  public void testBar() {
     System.out.println("bar");
  }

  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

then it gets executed. Here's what I've done:

• verified Maven version: Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
• verified Surefire version: followed this advice
• verified Surefire version: checked Surefire jars in my ~/.m2/repository/org/apache/maven/surefire -- all of them are either version 2.4.2 or 2.4.3
• done a mvn dependency:tree | grep junit to ensure I only depend on junit version 4.7

The module I am having this problem at doesn't have JUnit 3 tests.

Is there anything else I am missing?

Solution 1

mvn -X helped me to reveal the following:

...
[INFO] [surefire:test {execution: default-test}]
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3:runtime (selected for runtime)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:runtime (selected for runtime)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-booter/2.4.3/surefire-booter-2.4.3.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-api/2.4.3/surefire-api-2.4.3.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.testng:testng:jar:jdk15:5.8:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/testng/testng/5.8/testng-5.8-jdk15.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] Retrieving parent-POM: org.apache.maven.surefire:surefire-providers:pom:2.4.3 for project: null:surefire-testng:jar:null from the repository.
[DEBUG] Adding managed dependencies for unknown:surefire-testng
[DEBUG]   org.apache.maven.surefire:surefire-api:jar:2.4.3
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3
[DEBUG]   org.codehaus.plexus:plexus-utils:jar:1.5.1
[DEBUG]   org.apache.maven.surefire:surefire-testng:jar:2.4.3:test (selected for test)
[DEBUG]     org.apache.maven:maven-artifact:jar:2.0:test (selected for test)
[DEBUG]       org.codehaus.plexus:plexus-utils:jar:1.0.4:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG]     org.testng:testng:jar:jdk15:5.7:test (selected for test)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:test (selected for test)
...
[DEBUG] Test Classpath :
...
[DEBUG]   /home/mindas/.m2/repository/junit/junit/4.7/junit-4.7.jar

So it seems that the problem was coming from testng jar requiring JUnit v3.8.1. Even though Test Classpath was set to depend on JUnit 4, it was too late.

testng dependency was located in my POM:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.8</version>
  <scope>test</scope>
  <classifier>jdk15</classifier>
</dependency>

Immediately after I have commented it out, tests started to execute.

Lessons learned:

• mvn dependency:tree is not always enough, mvn -X is a friend.
• surefire is not made for developer heaven (I have realized this while looking at project JIRA reports). This is especially true as there are no other alternatives if you use Maven.

Thanks everybody for your help. Unfortunately there is no way to split answer points between Pascal and Kaleb, but Kaleb's advice to use mvn -X helped me to get on the right track so correct answer points go to him.

Solution 2

The Surefire plugin figures out which JUnit provider should be used based upon the classpath. If there are multiple JUnit versions on the classpath, you can either correct the classpath to have only one JUnit version on the classpath (as discussed above), or you can explicitly specify which provider you want to use. For example, specifying the following in your (parent) POM forces using the newest provider (e.g., "surefire-junit47"):

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <!-- Force using the latest JUnit 47 provider -->
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

Note however that Surefire 2.7 changed the way it's determining which unit test classes are run. The new behavior when using Surefire 2.7 (or later) with JUnit 4 is that any test without a @Test annotation will be skipped automatically. This may be great if you just have JUnit 4 unit tests, but if you have a combination of JUnit 3 and 4 unit tests, using the "surefire-junit47" provider will not work correctly. In such cases, its best to explicitly choose the "surefire-junit4" provider:

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <!-- Use the older JUnit 4 provider -->
      <artifactId>surefire-junit4</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

Solution 3

I don't know what you mean by "can't execute," but does it help to explicitly set the includes used by the maven-surefire-plugin?

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
</plugin>

Also, does running maven with the -X flag provide any useful information?

Solution 4

One more possible cause can be this bug (closed with "Won't fix"): https://issues.apache.org/jira/browse/SUREFIRE-587

Short summary: Tests extending TestCase (but not using annotations) will not be picked up if their name does not end with "Test".

Solution 5

For some poor soul out there who is wondering why Maven doesn't pick up JUnit tests.

I have both JUnit and TestNG as dependecies. But I want failsafe to run my functional tests using TestNG and surefire to run my unit tests using JUnit.

However, I found that surefire was attempting to run my unit tests using TestNG and found nothing to run. My JUnit tests were skipped.

Later I came across this Maven issue and configured surefire to run only "JUnit" tests like this:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <properties>
      <property>
        <name>junit</name>
        <value>true</value>
      </property>
    </properties>
  </configuration>
</plugin>

Hope this helps someone.

View more solutions

Comments

[Andreas Dolk]

Could you post your POM? That would be easier.

[mindas]

I can't post entire POM for both legal reasons and length - it's a big project, with hundreds of lines just in the POM. I could probably post relevant part(s), but not sure what exactly can be relevant.

[Devanshu Mevada]

Well, at least the maven-surefire-plugin configuration (if you have one or if you are inheriting one). Can you say if this problem is specific to one module? Does it happen outside the project? Are you inheriting from a corporate POM?

[Kaleb Pederson]

Can you provide detail on what the symptoms of "can't execute" are?

[mindas]

Yes, all modules have these settings. Thanks for your input, though!

[Devanshu Mevada]

The test would fail without 1.5+ compiler level, but it would be picked up.

[Andreas Dolk]

I know that it's just wild guessing - I never had trouble with Junit4 tests in maven...

[mindas]

I did what you suggested and indeed both tests (AppTest and my test were executed). Will try to dig this deeper tomorrow and/or post the relevant POM bits. Thanks a lot Pascal! Your method could benefit others while debugging same problem, too. Btw, can you modify your plugin mvn code - you forgot to close plugin tag at the end.

[Devanshu Mevada]

Me neither. I'm really wondering what could cause this weird behavior.

[mindas]

Thanks for your tip to use -X, this helped me to reveal the problem (see my own answer)

[Devanshu Mevada]

Actually, you should accept your own answer as it is providing the whole solution to your problem:)

[AHungerArtist]

Thank you so much for this tip. I had the same issue and it was because of PowerMock. Again, big help as without this post I would have been looking in the wrong direction for some time.

[mindas]

No, this wasn't the case for me (see my own answer for the real cause). Anyway, thanks for the tip - might be helpful for other people.

[Gábor Lipták]

Yeah, I saw your solution. I had the same question, and for me the solution was this bug. Thats why i shared it here.

[Gal]

Thanks! I spent several hours debugging because of this "feature".

[ocroquette]

Updated link for SUREFIRE-587 : issues.apache.org/jira/browse/SUREFIRE-587 Thanks for the hint Gábor.

[Kushal Paudyal]

This particular solution helped me !!

[Akshay Hazari]

Debugging shows it pulls junit-3.8.1.jar transitively. How do I override it.

[Visarut Sae-Pueng]

Noted: I upgrade my service from 2.3.5.RELEASE to 2.6.7 - Add dependency junit 4.13.2 and junit-vintage-engine - Swagger 2 change to use springdoc-openapi-ui 1.6.8 - management.info.env.enabled=true for path /info