Interpreting the ActiveMQ 5.3.0 SpecJMS2007® Result

Hot on the heels of the latest Apache ActiveMQ release, official SpecJMS2007® Results appear. ActiveMQ does 156 vertical and 60 horizontal. But what does that mean?

Some Background
SpecJMS2007® is a representative, long-running, comparative test. Let me take each of these in turn.

By representative, I mean that it contains a mix of business interactions that utilises point to point (or queue semantics) and publish/subscribe (topic) semantics. The message sizes vary within limits using random generators and there is a mix of non-persistent and persistent messages. All persistent messages are delivered and consumed within transactions. In reality, the interactions are based around a supermarket supply chain application which provides a rich tapestry for realistic actor interplay. Supermarkets querying suppliers, suppliers interacting with distribution centers and throughout, management in headquarters, keeping track of all dealings.

By long running, I mean that the scenario lasts a minimum of 30 minutes, excluding a warm-up period. The verification phase is based on periodic throughput and response time sampling during that period. In this way, the test verifies the sustainable load characteristics of a JMS Broker.

By comparative, I mean that the test artifacts and environment are completely specified such that results are totally reproducible. For example, if the broker implementation is swapped out from the ActiveMQ submission bundle, a comparable result for the same platform can be obtained. The platform (or OS and hardware configuration) must be maintained to produce comparable results. This focuses the comparison on the implementation of the broker, which is the intention.

Explain the Numbers

The numbers are seed values. They provide the base value or multiplier on which subsequent decisions like the number of destinations, quantity of messages etc. are determined for a given test run. An increase in the seed value has a cascade effect on the overall load that is placed on the system. If the seed value is too high, the overall load will result in a failed test. Either because of unacceptable throughput variance or because of response times exceeding predetermined ranges. To pass the test for a given seed value, all response time and throughput expectations must be met.

The vertical and horizontal qualifiers refer to the SpecJMS2007® workload topologies. The topologies are not directly comparable because the seed multipliers have different effects in both topologies.

Vertical
As the seed value increases, the vertical scenario aims to increase the number of messages that are processed for a given number of destinations. So in the supermarket supply chain parlance, this means increasing the quantity and variety of stock that is maintained and the frequency of replenishment of said stock. The number of supermarkets and suppliers etc. remains constant as the base seed value increases. In this way, the ability of the broker to deal with increased load on existing destinations is explored. Another way of looking at this is that the depth of the destinations rather than the number of destinations is increased.

Horizontal
As the seed value increases, the horizontal scenario aims to increase the number of destinations while using a fixed load of messages. This corresponds to adding more supermarkets, suppliers and distribution centers. The quantities of stock and the frequency of replenishment is constant. In this way, the ability of the broker to deal concurrently with large numbers of destinations is explored.

In short, the numbers are meaningless in isolation as they are the units of SpecJMS2007® performance measurement and these units have no real-world corollary. Where they are useful is when used in comparison with another run of the SpecJMS2007® test using a different JMS implementation or with some broker configuration tweak.

For example, there are two platform variants of the results, one with Hyper Threading(HT) enabled and the other with HT disabled. The effect is significant indeed, with the vertical successful seed value going from 138 to 156 and horizontal from 52 to 60. So turn HT on!

Apache ActiveMQ Out Of Memory!

Apache ActiveMQ is adaptable and configurable. A large part of its popularity is due to its flexibility. However, it comes with a default activemq.xml configuration file that cannot possibly suit everybody's needs. The default configuration is a compromise between memory utilisation, low latency and high throughput, with a smattering of feature demonstrations. In all, it is probably too much for one configuration file, but that is another issue that is in part addressed in version 5.3.0.

With the current defaults, it is relatively easy to push the broker's heap memory utilization past the -Xmx512m heap limit passed to the JVM in the start script. When that happens the broker begins to fail in various places with java.lang.OutOfMemoryError: unable to create new native thread or java.lang.OutOfMemoryError: Java heap space.

What to do?
Well Google is your friend but there is also the ActiveMQ FAQ and particularly the entry that deals with the likely causes and relevant configuration that can alleviate ActiveMQ OutOfMemoryError Exceptions.

In short, the answer is nearly always configuration and the intent is that the OutOfMemory FAQ entry will provide a comprehensive reference for the relevant options. Let it be your first port of call.

Building activemq from source with m2eclipse on Mac OsX

On Mac OsX, when building activemq from source using the neat m2eclipse import maven project feature, the activemq-fileserver module fails with error:

'Access restriction: The type HttpURLConnection is not accessible due to restriction on required library /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Classes/classes.jar'.

This is in fact, a reasonable error. It is not desirable to have a dependency on a sun internal class. When using a sun jdk it is not really an issue but the default eclipse java builder on OsX is using the Apple JVM and the java builder is correctly configured to consider this sort of reference an error. To get a clean build this error needs to be reduced to a warning.

The option to disable is at:

Eclipse -> Preferences -> Java ->
Compiler ->
Errors/Warnings ->
Deprecated and restricted API ->
Forbidden reference (access rules)

Change the drop down selection from Error to Warning.

Should this be fixed? Yeah, we should probably depend on commons http client instead. I will need to dig a little further to understand why access to the implementation class is needed in the first place.
Short term, suppressing this error allows the build to proceed.

activemq systemUsage xml configuration and sendFailIfNoSpace ...

I was caught out with this twice in 24hrs. The systemUsage sendFailIfNoSpace attribute must be configured on the XBean element content, not on the element wrapper.
The use case is to limit the pending message length by memory usage and to fail the producer with an exception when the memory limit (or 70% of the memory limit) is reached. The limit can be reached very easily with a fast producer and slow (or no) consumer.

In activemq XML configuration use:

<systemUsage>

 <systemUsage sendFailIfNoSpace="true">

   <memoryUsage>

     <memoryUsage limit="20 mb">

   </memoryUsage>

 </systemUsage>

</systemUsage>

If the attribute is incorrectly added to the top level element, it is ignored and the result is that a producer will experience the default "wait for space to become available" behaviour and will hang.
Note to self, be sure to double check where XBean attributes are specified!

Speaking at IJTC 2008 - Choosing a JMS

I will be presenting at IJTC 2008 next week. The focus of my talk will be on choosing a java messaging solution with an emphasis on making a decision in context. Trying to make all the goodness of the web and community work for you. It should be fun.