How can I monitor memory usage for a windows-based JVM and trigger an alert if it gets too high?

Question

I have an app that runs as a JVM in Windows 2008. I'd like to monitor it's memory usage and be sent an email if/when memory usage reaches a specific threshold. Is there a free utility or method for doing this?

Solution 1

The Win2008 OS can send alerts for performance counter threshold conditions.

Part 1 # Perfmon
Launch Performance Monitor (perfmon.msc)
Create a new User Defined Data Collector Set
Choose to create manually after naming
Select Performance Counter Alert
Click Add - choose the Object, Counter and Instance to monitor. Process/Java.exe/Working Set
Specify the alert threshold
Choose "open properties for this data collector set"
Specify a start schedule if the server is on a reboot schedule
Change the sample interval. Alert repeats at this interval when value it above threshold
Under Alert task, give it a name (like sendMail)
Start the Data Collector Set

Part 2 # Schedule Tasks
Open up scheduled tasks (compmgmt.msc/configuration/task scheduler)
Create a task, not a basic task
Name it the exact same name you did in the above section (like sendMail)
Check "user is logged in or not" so that it runs all the time. Change username if necessary.
Create a new email action under the Action tab
Enter all the info for from, to, subject, etc.
Enter the SMTP server.
You will be prompt for the password to run the task when you are not logged on.

Solution 2

In fact JVM memory boundaries are controlled by startup parameters like -Xmx2048m -Xms256m -XX:MaxPermSize=512m (it's example). JVM will use that allowed memory. Stable not leaking application will not go out of that boundaries.

Back to monitoring, it is possible to turn on JVM internal SNMP agent and then monitor memory usage of heaps and PermGen and through private SNMP community you can setup thresholds when exceeded JVM will send SNMP traps. In fact it is "enterprise solution of monitoring" by tools like HP OpenView. Cmdline parameters involved:

-Dcom.sun.management.snmp.port=10001 -Dcom.sun.management.snmp.acl=/home/liferay/snmp.acl -Dcom.sun.management.snmp.interface=0.0.0.0

They are selfexplanatory. ACL file template is at JRE_HOME/lib/management/snmp.acl.template. To get readable anwers not just SNMP OIDs, download JVM-MANAGEMENT-MIB.mib. Some short explanation is here on Oracle site. When SNMP enabled, you can check returned gauges by running:

snmpwalk -v2c -c public YourWinHost:10001 -m JVM-MANAGEMENT-MIB jvmMgtMIB

It expect you already put JVM-MANAGEMENT-MIB.mib to default Net-SNMP MIB directory (Unix or CygWin commandline).

Not to forget mail sending, I'm doing JVM monitoring by Zabbix in conjunction with SNMP on cluster running nationwide backend cluster of Post Office. It covers all run time monitoring requirements. Zabbix can send you email or Jabber IM message in case of some trigger fired. Zabbix support SNMP agent monitoring and SNMP traps too. I ran month test of services in cloud, when monitored JVM was somewhere in USA and Zabbix server in Ireland... it worked well even for such distances/delays. Since version 2.0 there is JMX monitoring possible, but for secure environments it could be complicated, cause JMX-RMI setup some ports random.

In general SNMP is enough for JVM run time monitoring and JMX can more help on development/test machines cause it can monitor application nuances.

If you are in cloud with your Windows machine, you can use one of SaaS monitoring services. I was pretty impressed with NewRelic services. It is easy to setup and really helpful for development/test phases, cause can do things like thread dumps, slow SQL queries a.s.o. It can send emails when threshold reached.

Solution 3

Since Windows 2012 Server deprecated the "send email" task type you will now need a script that will send an email. For example, a VBScript:

Set Msg = CreateObject("CDO.Message")

With Msg

 .To = "[email protected]"
 .From = "[email protected]"
 .Subject = "low memory"
 .TextBody = "low memory, check the server now"
 .Send
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2 
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpserver") = "mail.yourserver.com"
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpauthenticate") = cdoBasic
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = 25
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpusessl") = False
 .Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/sendusername") = "smtp-user"
 .Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/sendpassword") = "smtp-password"

End With

The other steps are pretty much the same. Refer to this blog post for more info

Solution 4

You can run JMX, but it doesnt show e.g. garbage collector. You can also monitor the logfile. There is no utility to restart Java when it runs out of memory. But it's easy to code just in Java, and run as server, which checks for the apps if they are running OK, and the business logic you need to invent yourself, e.g. if(TotalCPUTime > 1d) restartJava();

Anyway, to restart Java remotely over JMX, requires access to the server, so there is no utility to do these two at the same time - to actually run, watch, restart and monitor the Java Process. You can run this Java on another Java as well, so it will do monitoring and restarting at the same time, and would run a small java.exe to do it.

Solution 5

You can setup monitoring with JMX.

Is this a server app running on Tomcat or a JEE application server? or is this a command line Java application? or a Windows Service Wrapped Java app? Which JVM? What settings? Oracle JDK Rockit has a great memory monitor, Oracle JRockit Mission Control 4.1 or http://visualvm.java.net/index.html.

You could write a thread within your app that sleeps and awakens to check memory.

See these posts: https://stackoverflow.com/questions/1058991/how-to-monitor-java-memory-usage or https://stackoverflow.com/questions/74674/how-to-do-i-check-cpu-and-memory-usage-in-java

View more solutions

Comments

[Lucky Luke]

This is definitely possible, but I'd like to point out here that setting up performance counter alerts/triggers is somewhat crude, since it will trigger an alert as soon as that threshold is met, even if it just met for 10 seconds. This may work somewhat well for counters that do not change often (like memory usage), but would not work very well for more volatile counters like CPU usage. Monitoring solutions like EventSentry are a bit smarter, and let you configure the time period over which the threshold has to exceed a limit before they trigger an alert. This helps reduce false positives.