Performance / Stress Testing Java EE applications

Question

It's difficult to find all bottlenecks, deadlocks, and memory leaks in a Java application using unit tests alone.

I'd like to add some level of stress testing for my application. I want to test the limits of the application and determine how it reacts under high load.

I'd like to gauge the following:

• Availablity under high load
• Performance under high load
• Memory / CPU / Disk Usage under high load
• Does it crash under high load or react gracefully

It would also be interesting to measure and contrast such characteristics under normal load.

Are their well known, standard techniques to address stress testing. I am looking for help / direction in setting up such an environment. Ideally, I would like to run these tests regularly, so that wecan determine if recent deliveries impact performance.

Solution 1

I am a big fan of JMeter. You can set up calls directly against the server just as users would access it. You can control the number of user (concurrent threads) and accesses. It can follow a workflow, scraping pertinent information page to page. It takes 1 to 2 days to learn it well enough to be productive. (You can do the basics within an hour of downloading!)

As for seeing how all that affects the server, that is a tougher question. I have used professional tools from CA and IBM. (I am drawing a blank on specific tool names - maybe due to PTSD!) I have used out-of-the-box JVM profilers. I have used native linux and windows tools. If you are not too concerned about profiling what parts of your application causes issues, then you can just use the native tools for your OS to monitor CPU/Memory/IO.

Solution 2

One of our standard techniques is running stepped-ramp load tests to measure scalability.

Solution 3

There are mainly two approaches for performance on an application:

Performance test and System Test

How do they differ? Well it's easy, it's based on their scope, Performance tests' scope is limited and are highly unrealistic. Example: Test the IncomingMessage handler on some App X, for this you would setup a test which sends meesages to this handler on a X,Y,Z basis. This approach will help you pin down problems and measure performance of individual and limited zones on your application.

So this should now take you to the question, so am I to benchmark and performance test each one of the components in my app individually? Yes if you believe the component's behavior is critical and changes on newer versions are likely to induce performance penalties. But, if you want to get a feel on your application as a whole, the bunch of components interacting with each other and see how performance comes out, then you need a system test.

A system test will always, try to replicate as close as possible any customer production environment. Here you can observe what a real world feel of your app's performance is like and act accordingly to correct it.

So as conclusion,setup a system test on your app and measure what you were saying you wanted to measure. Then Stress the system as a whole and see how it reacts, you will be surprised on the outcome.

Finally, Performance test individually any critical components you have identified or would like to keep tracked on your app.

As a general guideline, when doing performance you should always: 1.- Get a baseline for the system on an idle state. 2.- Get a baseline for the system under normal expected load. 3.- Get a baseline for the system under stress conditions.

Keep in mind that Normal load results should be extrapolated to stress conditions, and a nice system will always be that one which scales linearly.

Hope this helps.

P.S. Tests, envirnoment setup and even data collection should be as fully automated as possible, this will help you run this on a basis and spend time diagnosing performance problems and not setting up the test.

Comments

[Rami]

I would recommend Apache JMeter jmeter.apache.org as an excellent stress testing tool for web applications. It is easy to use and can be extended as well.

[cmd]

I've found Gatling (gatling.io) to be an excellent tool as well.

[corlaez]

Regarding "It would also be interesting to measure and contrast such characteristics under normal load." I would invite you to check Metrics: metrics.dropwizard.io/3.1.0