“Perfbench” update: Tapestry 5 and Grails

I attempted to port the Seam “hotel booking” sample to Grails 1.1.1 as well as Tapestry 5.1, to add to this head-to-head comparison of Apache Wicket and JBoss Seam done earlier this year. You can find the code here: [browse] [SVN]

Here are the relative performance test results (page response time in milliseconds):

Heap-dump comparison for 20 concurrent user / sessions:

I’ll save detailed framework impressions and comparison for a later blog post, and I may need to make updates to the code based on feedback. Here are some overall observations:

• Grails was far more productive than Tapestry 5. This was mainly due to the documentation quality of Grails compared to the scattered and not very well organized Tapestry documentation.
• Ease of writing custom tag-libraries is IMO one of the best things about Grails.
• The two missing rows in the Tapestry performance test results are because I gave up trying to implement form field validation over Ajax when the user “tabs out”. Actually I didn’t implement form-field validation at all because I could not figure out how to integrate Hibernate Validator. Maybe I missed something obvious, do let me know.
• Grails still has some way to go in terms of performance. I am told that significant performance optimizations for GSP will make it into 1.2
• Overall, Wicket is fastest, with Tapestry coming a close second.
• Wicket also takes up the least amount of heap. 31 MB of the Grails heap alone is taken up by instances of the “groovy.lang.ExpandoMetaClass”.
• Session usage of the Seam + JSF combination is significantly higher compared to all the rest, around 760 KB per session.

Thoughts on Tapestry 5

I was very interested in seeing if Tapestry 5 lived up to its promise of significantly better performance and scalability – which is known to come at a cost: a programming model that treats web-pages as static structures that can be “pooled”. One example of what this means is that you can’t rely on constructors anymore so you now have to figure out other ways of re-initializing your server-state (if applicable) after you are handed an instance from the pool. The “magic” of getting a page from the pool or getting hold of request or session scoped variables is all achieved predictably – using annotations.

Another example: due to the way Tapestry does byte-code manipulation – the annotation approach works only for “private” fields and not “protected” ones. I found this out the hard way, encountering an exception when I tried to extend from an abstract base class and re-use code the “old-fashioned” object-oriented way.

There’s a lot more I can go into, but for now, to summarize my experience as a long-time Wicket user trying out Tapestry 5 – the top few differences I found are:

• Having to work around the implications of Tapestry pages being static and “annotation driven” programming like I described above.
• Tapestry does not have the equivalent of Wicket Model-s which give you fine-grained control over the data you need to render (and bind) within a page and how much of it you decide to persist in the session. Model-s are certainly the aspect of Wicket that takes the most getting-used-to, but I have a new-found respect for them now after trying Tapestry.
• I would rate the Ajax infrastructure of Wicket higher: for example Wicket Behavior-s are cleaner compared to Tapestry Mixin-s, and calling / handling Ajax requests (especially when you depend on server-side state) is far easier.
• Tapestry 5 does not have built-in support for web-flow or “conversation scope” so I had to write some code to maintain a map of bookings in the session. BTW I did not have to do this for Grails as I could use WebFlow.

So coming back to the question, does the “Tapestry way” have a distinct advantage and are the quirks of the programming model worth it? The opinion of the Tapestry team on this is pretty clear. Take this quote from the Tapestry home page for example:

In some Tapestry-like frameworks, such as Faces and Wicket, the page structure is more dynamic, at the cost of storing much, much more data in the HttpSession.

Well, looking at the performance comparison results, my personal conclusion is that statements like the above are incorrect. In fact Tapestry seems to take up more heap space than Wicket for the same functionality. Also worth noting are the results of running the Wicket application using the HttpSessionStore instead of the default SecondLevelCacheSessionStore + DiskPageStore:

Because this time, Wicket is faster now even for the “post confirm booking” page. This particular action actually ends up displaying 200 items in an HTML table by the end of the test run for 20 concurrent users, since pagination is not being used.

The benchmark has been designed to be easy for you to run once you check-out the code, refer the end of this blog post for details. The details of the environment I used for the results posted here are as follows:

• java.runtime.version : 1.6.0_16-b01
• java.vm.name : Java HotSpot(TM) Server VM
• java.vendor : Sun Microsystems Inc.
• os.name : Windows XP
• os.version : 5.1
• sun.os.patch.level : Service Pack 3
• Intel Core 2 Duo 3.01GHz
• 3 GB RAM

Feel free to post your results or suggest changes to the code.