Dmytriy Kobyleha and I have a small application on Facebook. We’re developing it to check our ideas or make some kind of experiments. We are using java webapp based on Spring, Hibernate, Lucene. As you may know, Facebook platform has strict requirements to applications. One of them – your response time should be less than seven seconds. So you should care about your application performance and measure it precisely.

One day Dmytro Shteflyuk told me about a new service from New Relic. New Relic is the great performance monitor for ruby on rails apps. Since Sun released JRuby, and added support of Ruby on Rails apps to GlassFish, a lot of Ruby developers started using this software. I think that New Relic decided to build a java performance monitor because of those developers. Recently they’ve launched a beta service – NewRellic for Java Apps. So I decided to install and use it for a couple of weeks.

I’ve signed up to New Relic and received instructions on how to install it on my system. It looks very easy. New Relic has implemented own java agent and all I needed were to download a distribution and configure environment variable:

export JAVA_OPTIONS="${JAVA_OPTIONS} javaagent:/path/to/newrelic/newrelic.jar"
At this point I encountered a problem. I used jsvc to run Tomcat as a daemon and it doesn’t support -javaagent option. I spent some time to reconfigure my tools to run Tomcat the usual way. As soon as I fixed it everything started working perfectly. I didn’t change anything in my project. And it’s cool. I was very busy and left it alone.

I came back a week later. I saw a lot of data and graphs related to my application. First I looked at Apdex graph.

apdex graph
apdex graph

Apdex score is a complex value which shows you how well your application performs. I enjoy it very much. If you will look at the image again once more thoroughly then you will realize that my application stumbles on some problem each 15 minutes. One graph showed me that. Isn’t it cool??

I started investigating my problem. I opened the Memory section. Here you can find memory pools, CPU, and Garbage collection statistics. The last ones were very interesting for me.

memory graph
memory graph

My fear about the “15 minutes” problem became more stronger. I looked at the Database section. And….

database graph
database graph

I found my problem! I just realized why it happened each 15 minutes. We have a cron job to calculate statistics, started every 15 minutes. I’ve never thought that this job could impact the application performance so much. Finally, I saw Transaction Traces logs.

transaction traces
transaction traces

I like it very much. I can see which action has long response time. As you can see our job takes 10 seconds to complete. Another cool graph is Transactions. Well, I think New Relic should rename it because I thought about database transactions at first.

transactions
transactions

This graph shows you responses and average response time per small period of time. Also you could find the slowest actions on this page. It’s really cool.

Of course, there are a lot more charts which didn’t give me useful information. Maybe I don’t understand it – who knows? :) Also, New Relic should improve calendar for compares. It’s totally unusable now. Anyway, I like this software and will continue using it.

P.S. We have 0.95 Apdex now. I wanna 1.0 so I’m still investigating our problems :).