How to separate unit testing and integration testing on a maven project

Question

I have a maven project and lots of junit classes in it. I develop with Eclipse. I want to separate functional test classes and integration testing classes.

When I build the project in Eclipse then I want only the functional test classes to be executed.

By jenkins both of them should be executed.

Which approach should i follow?

Solution 1

I find it more convenient to put integration tests in separate projects, and then run them as if they were unit tests by relying on Maven's default life cycle. As I have to run my tests against different environments, this approach makes it easier to manage environment specific tests.

Let's assume I have an application, represented by the application Maven aggregator project, which contains a jar module called project. I keep unit tests within project itself, so that they get executed whenever I build my application. This is also built every night by Jenkins; ideally successful builds should be automatically deployed to one or more test environments, for both manual and automatic tests. Currently this is done by hand.

For every environment where I need to run my integration tests I have an applicationTestEnvX Maven aggregator project. This contains at least a projectTest module, where I keep those integration tests that are environment independent, as well as any test support code. Tests for my project module that are specific to environment X are kept in a projectTestEnvX module. I have a Jenkins job for each applicationTestEnvX project, which runs my tests every night. Ideally these should be run against the result of the application build, but I'm not there yet.

There is also a direct correspondence with how my projects are stored in Subversion and my Eclipse workspaces, but that's another story ;-)

Solution 2

Have a look at two Maven plugins: Surefire (for unit tests) and Failsafe (for integration tests). They closely resemble each other, Failsafe is a clone of Surefire.

Organize your tests so that their naming schema goes with proposed configuration: **/*Test.java for unit tests and **/*IT.java for integration. Surefire is run by default, for Failsafe you'll need extra excerpt in POM — example and more info in this answer.

Then it's down to mvn test or mvn integration-test.

If you want to run integration test only in certain environments (Jenkins), you could make Failsafe executing only in a profile, for example:

<profiles>
    <profile>
        <id>env-itest</id>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                 <executions>
                    <execution>
                        <id>integration-test</id>
                        <goals><goal>integration-test</goal></goals>
                    </execution>
                    <!-- other executions, if needed -->
                 </executions>
            </plugin>
        </plugins>
    </profile>
</profiles>

Then, on Jenkins, you run mvn clean install -P env-itest and on your local environment only mvn clean install (or simliar).

Solution 3

Have a look at the maven documentation around the integration test phase - you can use different plugins to control which tests are run, simply by naming the tests appropriately.

Then running mvn test will build & run your code & unit tests, which mvn verify will run your integration tests as well.

Solution 4

You can separate your unit and integration tests into separate packages (or perhaps even separate source folders, but then you'd have to updated your Maven configuration to recognize that you have two separate source folders for tests).

To take advantage of this, in Eclipse's Run Configurations (Run > Run Configurations), create a new JUnit run configuration that "Run all tests in the selected project, package or source folder:", select the the package/source folder containing only the tests you want to run.

---

When I first read your question, I got it backwards. I thought you wanted to run the full suite in Eclipse, and only a subset in Jenkins. I'm going to just leave my old answer up in case you find this useful some how:

The way I've done this before is through naming convention of the JUnit Test Cases.

I would name all the unit test test cases ...UnitTest (e.g., RegistrationManagerUnitTest) and integration test test cases, I'd name ...IntegrationTest (e.g., RegistrationDaoIntegrationTest).

Then in Maven, you can configure it to run all the test cases whose classes end with ...UnitTest (by default it's looking for classes whose name end with ...Test. Something along the lines of:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <!-- Run only tests whose name end with "UnitTest" -->
        <includes>
            <include>**/*UnitTest.java</include>
        </includes>
    </configuration>
</plugin>

Solution 5

FYI, with TestNG, you would simply use groups, e.g. @Test(groups = "integration") and @Test(groups = "unit"). Then you simply run different groups depending on what you need.

View more solutions

Comments

[Kayser]

Actually, Functional test are the unit test without requiring any other class or system. Integration test include in my case also the communication with other systems etc. The goal is to do these tests only on continous integration server namely jenkins.

[Kayser]

It is really an interesting approach that you mention. But how about Jenkins. How can they communicate with each other. What I mean. Should i create three projects on jenkins for each of them (Ori-Project, UnitProject, IntegrationProject)? Can you explain a little detailed..?

[MaDa]

@Kayser See my expanded answer.

[Kayser]

How does failsafe know that it execute only the test ending with IT?

[Kayser]

I get this error in Pom if I add build part into profiles. <br/><br/>cvc-complex-type.2.4.a: Invalid content was found starting with element 'groupId'. One of '{"http:// maven.apache.org/POM/4.0.0":plugin}' is expected.

[Kayser]

In that case I would have three modules (Parentmodule: MyProject, Child1Module: UnitTestProject, Child2Module: IntegrationtestProject). Right??

[Nicola Musatti]

Aggregator and parent should be distinguished: an aggregator is a container for modules that need to be built together; a parent project is a means for sharing settings among projects. In my example application is an aggregator that contains the project module; applicationTestEnvX is another aggregator that contains the projectTest and projectTestEnvX modules. You might create a third aggregator for unit tests, but I don't see a need for it; if tests are fast they can remain in the main project, otherwise move them to the integration project.

[Kayser]

In my case i must seperate integration tests and unit tests (organisational reasons.) It means: according to your explanation I need four projects. - Aggregatorproject - Myproject - IntegrationProject - UnitTestProject

[MaDa]

@Kayser 1. See Failsafe documentation (maven.apache.org/plugins/maven-failsafe-plugin) for exact info; it's just built into the plugin, but you can change it. 2. You're right, I pasted too much code. Fixed now.

[Nicola Musatti]

If your application fits in a single Maven project you don't need an aggregator, otherwise I'd expect you to have an aggregator for your source projects, one for your unit test projects and another one for your integration projects.

[Kayser]

Hi Cedric Beust. The Guru of TestNG. I must say TestNG has impressed me very much. The only problem of TestNG is the plugins by Eclipse and Netbeans. They are not mature enough. Therefore we chose to use JUnit in my company.

[Lemon]

Think you're supposed to run mvn verify and not mvn integration-test.

[Jonatan Cloutier]

it's is now possible to do the same with junit categories

[Balaji Boggaram Ramanarayan]

I feel its odd to have suffixes as *Test.java and *IT.java. The better approach would be to group them as "unit" or "integration" (Only if you are using testNG)

[Balaji Boggaram Ramanarayan]

Is it a good/best practice to have suffixes like : *UnitTest and *IntegrationTest. I feel this can be overcome using categaorization feature in to groups. I think both testNG and junit are supported with grouping.