SonarQube - Java - Sonar Scanner not analyzing src/test/java folders

Question

I downloaded SonarQube 7.0, and setup the Maven project to use Sonar scanner. The pom.xml looks like the following:

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.github.jitpack</groupId>
<artifactId>maven-simple</artifactId>
<version>0.2-SNAPSHOT</version>
<packaging>jar</packaging>

<name>Simple Maven example</name>
<url>https://jitpack.io/#jitpack/maven-simple/0.1</url>

<properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <mavenSurefireVersion>2.20</mavenSurefireVersion>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <src.dir>src/main/java</src.dir>
    <sonar.sources>pom.xml,src/main/java</sonar.sources>
    <mavenSurefireVersion>2.20</mavenSurefireVersion>
</properties>

<dependencies>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.8</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<profiles>
    <profile>
        <id>sonarLocal</id>
        <activation>
            <activeByDefault>false</activeByDefault>
        </activation>
        <properties>
            <sonar.host.url>http://localhost:9000</sonar.host.url>
        </properties>
    </profile>
</profiles>

<build>
    <sourceDirectory>${src.dir}</sourceDirectory>
    <testResources>
        <testResource>
            <directory>${project.basedir}</directory>
            <filtering>true</filtering>
        </testResource>
    </testResources>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.3</version>
            <configuration>
                <source>${maven.compiler.source}</source>
                <target>${maven.compiler.source}</target>
                <encoding>${project.build.sourceEncoding}</encoding>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>3.0.2</version>
            <configuration>
                <encoding>${project.build.sourceEncoding}</encoding>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-checkstyle-plugin</artifactId>
            <version>3.0.0</version>
            <executions>
                <execution>
                    <id>validate</id>
                    <phase>validate</phase>
                    <configuration>
                        <includeTestSourceDirectory>true</includeTestSourceDirectory>
                        <configLocation>google_checks.xml</configLocation>
                        <encoding>${project.build.sourceEncoding}</encoding>
                        <consoleOutput>true</consoleOutput>
                        <failsOnError>true</failsOnError>
                        <failOnViolation>true</failOnViolation>
                        <violationSeverity>warning</violationSeverity>
                    </configuration>
                    <goals>
                        <goal>check</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${mavenSurefireVersion}</version>
        </plugin>
        <plugin>
            <groupId>org.sonarsource.scanner.maven</groupId>
            <artifactId>sonar-maven-plugin</artifactId>
            <version>3.4.0.905</version>
        </plugin>
    </plugins>
</build>

My structure of the project looks like below:

src/main/java --> Contains app code.

src/test/java --> Contains all test code.

I use the command, "mvn clean install sonar:sonar -PsonarLocal" to execute Sonar.

Now I observed that Sonar never analyzes all .java files in the src/test/java folder.

To prove this, I added an unused variable in one of the test classes, and Sonar didn't catch it. If I do the same thing in any file in the src/main/java folder, Sonar catches it.

Is there something I'm missing?

Solution 1

Using the following commands, you can analyze code in both src/main/java and src/test/java in the same Sonar project:

1. First build your application and store its code coverage using JaCoCo:

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=false

2. Then upload the results to Sonar:

mvn sonar:sonar -Dsonar.host.url=http://localhost:9000 -Dsonar.login=<a valid token> -Dsonar.sources=src -Dsonar.test.inclusions=src/test/java/*

Note that the -Dsonar.login=<a valid token> property should only be necessary if http://localhost:9000 (or whichever host url) requires authentication. Replace with a valid token.

However, I recommend scanning your src/test/java code in a separate Sonar project, because metrics like code coverage get thrown off when scanning test code alongside application code (as Sonar will report 0% coverage on all the test classes themselves). To scan separately:

For application code (src/main/java), run:

mvn sonar:sonar -Dsonar.host.url=http://localhost:9000 -Dsonar.login=<a valid token>

For test code (src/test/java), run:

mvn sonar:sonar -Dsonar.projectKey=<my project>-tests -Dsonar.host.url=http://localhost:9000 -Dsonar.login=<a valid token> -Dsonar.sources=src/test -Dsonar.test.inclusions=src/test/java/*

Update -Dsonar.projectKey=<my project>-tests with the name of your project, so that you have a unique project key for the test scan.

For this approach, no additions specific to Sonar/JaCoCo need to be made to pom.xml.

Note that mvn clean with JaCoCo (step 1. above) must be executed before mvn sonar:sonar... whenever you make updates to your code in order for those updates to be reflected in Sonar.

I tested this using SonarQube 6.6 and Sonar Scanner 3.0.3.778.

Solution 2

You're expecting "normal" rules to be applied to tests. Today that's not the case. Instead, only a subset of test-specific rules are applied. Try instead marking a test @Ignored. You should see an issue then.

A way to double-check without another analysis is to look at the project's Code page. You should see the test files listed there and if you drill in to one should be able to access file-specific metrics from the "hamburger menu" icon at the top-right of the file viewer.

Currently, to run normal rules on source files, you would need to do a second, separate analysis (be sure to feed in a different project key with -Dsonar.projectKey or you'll overlay your normal analysis) and specify your tests directory as the sources (sonar.sources=relative/path/to/tests).