How to correctly configure the property "sonar.java.binaries"?

Question

We are using SonarQube 5.1.2 using Ant runner 2.2 and Java pluging 3.12 for the analysis. I can succesfully analyse my project. I just keep getting this error:

Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor@d678716, org.sonar.java.checks.unused.UnusedPrivateMethodCheck@58e28efd, CycleBetweenPackages rule are disabled.

So I need to configure my sonar.java.binaries and sonar.java.test.binaries properties (following http://docs.sonarqube.org/display/PLUG/Java+Plugin+and+Bytecode).

Which I think I have done correctly:

<property name="project.dir" value="${basedir}/xalg.prj/h3_service_fo" />   
<property name="sonar.java.binaries" location="${project.build.dir}/classes/main" />
<property name="sonar.java.test.binaries" value="${project.build.dir}/classes/test" />

Which resolve to the following valid directories for the above properties:

basedir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj
project.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main
sonar.java.test.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/classes/test

But I keep getting:

Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor@d678716, org.sonar.java.checks.unused.UnusedPrivateMethodCheck@58e28efd, CycleBetweenPackages rule are disabled.

And for the life of me, I can not figure out what values I need to give the sonar.java.binaries and sonar.java.test.binaries properties. I even tried using sonar.binaries, which gave me the following output:

Binary dirs: xalg.prj/h3_service_fo/build/classes

Which I did not get using either sonar.java.binaries or sonar.java.test.binaries. I also got:

JavaClasspath initialization...
sonar.binaries and sonar.libraries are deprecated since version 2.5 of sonar-java-plugin, please use sonar.java.binaries and sonar.java.libraries instead

Which is to be expected for a deprecated property. But using the sonar.java.binaries property I did not get the "Binary dirs" line in my log.

Using sonar.java.binaries:

Language is forced to java
Load rules
Load rules (done) | time=761ms
Code colorizer, supported languages: cs,plsql
Initializers : 
Base dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj
Working dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj\.sonar
Source paths: xalg.prj/h3_service_fo/src/main/java
Test paths: xalg.prj/h3_service_fo/src/test/java
Source encoding: windows-1252, default locale: en_US
Index files

Versus using sonar.binaries:

Language is forced to java
Load rules
Load rules (done) | time=736ms
Code colorizer, supported languages: cs,plsql
Initializers : 
Base dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj
Working dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj\.sonar
Source paths: xalg.prj/h3_service_fo/src/main/java
Test paths: xalg.prj/h3_service_fo/src/test/java
Binary dirs: xalg.prj/h3_service_fo/build/classes
Source encoding: windows-1252, default locale: en_US
Index files

I also looked into the source code of SonarQube, SonarQube Java Plugin and the SonarQube Scanner for instances of either "Java bytecode has not been made available to the analyzer." or sonar.java.binaries. I found plenty on sonar.java.binaries, but nothing on "Java bytecode has not been made available to the analyzer." So I have no clue what conditions exactly trigger that error.

I also tried the following permutations on sonar.java.binaries:

<property name="sonar.java.binaries" location="${project.build.dir}/classes" />
<property name="sonar.java.binaries" location="${project.build.dir}/classes/main/nl" />

But that did nothing either.

What is weird is that Squid seems to resolve the classpath just fine:

----- Classpath analyzed by Squid:
D:\appl\BuildAgent\work\H3\src.prj\java.prj\xalg.prj\h3_service_fo\build\classes\main

So, what am I missing? What am I doing wrong? Thanks in advance.

Update 2016-09-08:
Removed the entire log, the post become to long.

A subset with the (I think) relevant paths:

project.build.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build
project.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo
project.src.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src

sonar.dir=D\:/appl/sonarqube-5.1.2
sonar.working.directory=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\.sonar
sonar.projectBaseDir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj

sonar.jacoco.reportPath=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/jacoco/test.exec
sonar.junit.reportsPath=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/test-results

sonar.sources=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src/main/java
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main

sonar.java.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_deploy/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/lib/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/provided/*.jar

sonar.tests=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src/test/java
sonar.java.test.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/classes/test
sonar.java.test.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_deploy/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/lib/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/provided/*.jar

The paths have exactly the same format as in my post. Could it be that the Sonar Ant runner can't figure out a path with both backslashes and slashes?

Update 2016-09-16:
Removed the entire log, the post become to long.

A subset with the (I think) relevant paths:

project.build.dir=xalg.prj\\\\h3_service_fo\\\\build
project.dir=xalg.prj\\\\h3_service_fo
project.src.dir=xalg.prj\\\\h3_service_fo\\\\src

sonar.dir=D\:\\\\appl\\\\sonarqube-5.1.2
sonar.working.directory=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\.sonar
sonar.projectBaseDir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj

sonar.jacoco.reportPath=xalg.prj\\\\h3_service_fo\\\\build\\\\jacoco/test.exec
sonar.junit.reportsPath=xalg.prj\\\\h3_service_fo\\\\build\\\\test-results

sonar.sources=xalg.prj\\\\h3_service_fo\\\\src\\\\main\\\\java
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main
sonar.java.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_deploy\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\lib\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\provided\\\\*.jar

sonar.tests=xalg.prj\\\\h3_service_fo\\\\src\\\\test\\\\java
sonar.java.test.binaries=xalg.prj\\\\h3_service_fo\\\\build\\\\classes\\\\test
sonar.java.test.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_deploy\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\lib\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\provided\\\\*.jar

Some paths have become relative, but I think that is because TeamCity changed the Ant file to the file in SVN. The sonar.java.binaries is absolute and it definitely points to the correct directory.

But I still get this error:

09:17:52.299 INFO  - Java Main Files AST scan done: 1579 ms
09:17:52.301 INFO  - 2/2 source files have been analyzed
09:17:52.305 WARN  - Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor@757a48f9, org.sonar.java.checks.unused.UnusedPrivateMethodCheck@1adf492b, CycleBetweenPackages rule are disabled.

The classpath is still interpreted just fine:

[sonar:sonar] 09:17:51.971 DEBUG - ----- Classpath analyzed by Squid:
[sonar:sonar] 09:17:51.972 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\xalg.prj\h3_service_fo\build\classes\main
[sonar:sonar] 09:17:51.973 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\batch.daemon.jar
[sonar:sonar] 09:17:51.974 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\buildinfo.jar
[sonar:sonar] 09:17:51.975 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h2_shared.jar
[sonar:sonar] 09:17:51.975 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_generator.jar
[sonar:sonar] 09:17:51.976 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_loadtest.jar
[sonar:sonar] 09:17:51.977 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_common.jar
[sonar:sonar] 09:17:51.977 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg.jar
[sonar:sonar] 09:17:51.978 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_dao.jar
[sonar:sonar] 09:17:51.979 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_mappers.jar
[sonar:sonar] 09:17:51.979 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_procedures.jar
[sonar:sonar] 09:17:51.980 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare.jar
[sonar:sonar] 09:17:51.981 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_dao.jar
[sonar:sonar] 09:17:51.982 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_mappers.jar
[sonar:sonar] 09:17:51.982 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_procedures.jar

Could the Sonar Ant runner have a problem with the escaped back slashes?

Solution 1

This is what I use, and it works.

sonar-scanner -Dsonar.projectKey=projectName
-Dsonar.gitlab.commit_sha=$CI_BUILD_REF
-Dsonar.gitlab.ref_name=$CI_BUILD_REF_NAME -Dsonar.sources=directory\src\
-Dsonar.java.binaries=.build\libs\

You need to point sonar.java.binaries to a directory that contains jars. I would't try to get fancy with wildcards and file extensions, that's not the same thing. You need a directory as an argument, not a regular expression for files.

I don't know everything about sonar scanner, but my configuration works.

Take my example into context, I use it in a gitlab runner pipeline. Your double backslashes should be fine, but you can always substitute them for forwards slashes in your config files.

Solution 2

Here are some options for a Gradle multi-project. I'm running in CI (Jenkins) but the principles apply.

Options:

1. Hand-maintain a hardcoded sonar.java.binaries entry, eg: sonar.java.binaries=subproject1/build/classes,subproject2/build/classes, (etc)

2. Set it to the top-level directory of the project:

   sonar.java.binaries=.

3. Use a script to generate your own props file at runtime, for example:

   cat > sonar-project.properties <<EOF
   sonar.projectKey=XXXX
   sonar.projectVersion=YYYY
   sonar.sources=$(find . -path '*/src/main' | xargs | tr ' ' ',')
   sonar.java.binaries=$(find . -path '*/build/classes/java' | xargs | tr ' ' ',')
   EOF
   

Note that the above also applies to sonar.sources, and it is likely going to need to be in sync with sonar.java.binaries. Having binaries be a superset of source seems to be OK. Obviously, if your sources value is too broad (ie includes tests, etc), you'll get more noise. The advantage of using a script is that you can customize these heuristics to your taste.

Solution 3

Based on the documentation at https://docs.sonarqube.org/latest/analysis/languages/java/

The purpose of sonar.java.binaries path is:

Comma-separated paths to directories containing the compiled bytecode files corresponding to your source files.

For Gradle, the default path is at build/classes, so I would set:

sonar.java.binaries=build/classes