Remote monitoring with visualvm and JMX

Question

I would like to monitor a remotely running java (spring boot) application with jvisualvm (or jconsole). When running locally, I can see the managed beans in both jvisualvm and jconsole. When running remotely I cannot connect. I tried it with several different java processes (e.g. with spring xd). Looking for answers here on SO and on Google did not help.

These are my JAVA_OPTS (on the remote host):

$ echo $JAVA_OPTS
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=192.168.59.99

Then I simply start the program as follows (this is for spring xd, but I experience the same problem with other java programs).

$ bin/xd/xd-singlenode

The server process seems to pick up the options:

$ ps -ef | grep single
vagrant  22938 19917 99 06:38 pts/2    00:00:03 /usr/lib/jvm/java-8- oracle/jre/bin/java -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=192.168.59.99 -Dspring.application.name=admin -Dlogging.config=file:/home/vagrant/spring-xd-1.1.0.RELEASE/xd/config///xd-singlenode-logger.properties -Dxd.home=/home/vagrant/spring-xd-1.1.0.RELEASE/xd -Dspring.config.location=file:/home/vagrant/spring-xd-1.1.0.RELEASE/xd/config// -Dxd.config.home=file:/home/vagrant/spring-xd-1.1.0.RELEASE/xd/config// -Dspring.config.name=servers,application -Dxd.module.config.location=file:/home/vagrant/spring-xd-1.1.0.RELEASE/xd/config//modules/ -Dxd.module.config.name=modules -classpath (...)

The java version on the remote host (ubuntu linux vm) is:

$ java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

The java version on the local machine (Mac OS) is slightly different:

$ java -version    
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)

In jvisualvm I add the remote connection as follows (tried both with ssl connection and without):

This is the error message jvisualvm gives me:

I can connect from the local host to the remote host with the command telnet 192.168.59.99:9010, when the remote process is running -- so this does not seem to be a firewall problem.

Any help is highly appreciated.

Solution 1

Please use the following JVM options :

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.59.99

In the Jconsole use the following to connect:

service:jmx:rmi:///jndi/rmi://192.168.59.99:9010/jmxrmi

Solution 2

Arnab Biswas's anwser not work in my case. After an hour of researching, I found out that JMX runs on top of RMI, and as such, there are 2 ports that JMX utilizes:

• The JMX connect port. (-Dcom.sun.management.jmxremote.port)
• The (infamously) roaming RMI data port. (-Dcom.sun.management.jmxremote.rmi.port)

RMI data port will open a random port start from 1099. By setting the port used by the RMI registry and the RMI server to the same port, tunneling will be much easier.

So I need to add -Dcom.sun.management.jmxremote.rmi.port=9010 to JVM options

And I need to use the following JVM options :

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.59.99

Read more:

• http://hirt.se/blog/?p=289
• https://realjenius.com/2012/11/21/java7-jmx-tunneling-freedom/
• https://medium.com/codefountain/monitoring-using-java-visualvm-a25203d36390

Comments

[Arnab Biswas]

Could you please try with the following Java options : -"Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.59.99" and also in the Jconsole use service:jmx:rmi:///jndi/rmi://192.168.59.99:9010/jmxrmi

[user152468]

You saved my day. This works. Can you post this as an answer, then I will gladly accept.

[Arnab Biswas]

Glad to hear that it worked! Posted as answer.

[Jack Zach Tibbles]

It seems like a bad idea to make this accessible without authentication?

[Kyrstellaine]

@JackZachTibbles pretty common, though, inside corporate firewalls

[Frédéric Camblor]

BTW, providing credentials through -D option makes it clearly readable as soon as a user has access to running process options (ps command on linux for instance) which is often possible as soon as you're able to connect through ssh to your server... so ... not sure if it brings a lot of security on the table

[Krzysztof Krasoń]

OK, so what's the difference from the options or connection that the author provided? Please show what was added/removed, without it is hard to make this fix for others.

[Arnab Biswas]

I am sorry. If the question is for me, I will not be able to reply. That was more than 4 years back. I have moved on and don't work in Java any more. :-(

[Tobias Gierke]

Thank you !!!! I was slowly going insane trying to get a remote connection to a firewalled host working... IMHO this should be the accepted answer. Seems like almost nobody tries to connect to a machine sitting behind a firewall so they all just suggest to set "com.sun.management.jmxremote.port" but nobody mentions "com.sun.management.jmxremote.rmi.port" ...

[Tobias Gierke]

Did not work for me because this answer does not mention that the connection needs a second (by default dynamically allocated) port to work. IMHO answer by thangdc94 should be the accepted one.

[Aadam]

Ubuntu behind fire wall, UFW jmx, visualvm this should be done.. adding this comment so that it could pop up in google search :d

[Risalat Zaman]

This should be the accepted answer