How to start and stop Jetty – revisited

I mentioned in my previous post that I would blog about some of the things I learnt while putting together the Seam / JSF versus Wicket “perfbench“.

A while back I posted about how to start and stop Jetty from Ant – useful for those using the Jetty downloaded distribution. In this post I show how to cleanly shutdown a Jetty instance started in “embedded” mode. This tip may be useful for those using Jetty in a continuous integration build – for e.g. when Selenium tests are involved.

Info on starting an embedded Jetty instance is out there but I was not able to find ways to cleanly shutdown – other than hacks like this.

Update 2009-04-07: just found a blog post by Stephen Haberman that has a detailed explanation of WAR-less Development with Jetty

Here’s the code to start Jetty. The trick is to spawn a thread with a socket listening on another port (8079 in this case) that we can connect to later:

package mypackage;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import org.mortbay.jetty.Connector;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.bio.SocketConnector;
import org.mortbay.jetty.webapp.WebAppContext;

public class Start {

    private static Server server;

	public static void main(String[] args) throws Exception {
		server = new Server();
		SocketConnector connector = new SocketConnector();
		connector.setPort(8080);
		server.setConnectors(new Connector[] { connector });
		WebAppContext context = new WebAppContext();
		context.setServer(server);
		context.setContextPath("/wicket-jpa");
		context.setWar("src/main/webapp");
		server.addHandler(context);
        Thread monitor = new MonitorThread();
        monitor.start();
        server.start();
        server.join();
	}

    private static class MonitorThread extends Thread {

        private ServerSocket socket;

        public MonitorThread() {
            setDaemon(true);
            setName("StopMonitor");
            try {
                socket = new ServerSocket(8079, 1, InetAddress.getByName("127.0.0.1"));
            } catch(Exception e) {
                throw new RuntimeException(e);
            }
        }

        @Override
        public void run() {
            System.out.println("*** running jetty 'stop' thread");
            Socket accept;
            try {
                accept = socket.accept();
                BufferedReader reader = new BufferedReader(new InputStreamReader(accept.getInputStream()));
                reader.readLine();
                System.out.println("*** stopping jetty embedded server");
                server.stop();
                accept.close();
                socket.close();
            } catch(Exception e) {
                throw new RuntimeException(e);
            }
        }
    }

}

The “MonitorThread” in the inner class above stops the embedded Jetty server if a line feed is received. So the code to stop Jetty is pretty simple. Here we duly send “\r\n” to port 8079:

package mypackage;

import java.io.OutputStream;
import java.net.InetAddress;
import java.net.Socket;

public class Stop {

    public static void main(String[] args) throws Exception {
        Socket s = new Socket(InetAddress.getByName("127.0.0.1"), 8079);
        OutputStream out = s.getOutputStream();
        System.out.println("*** sending jetty stop request");
        out.write(("\r\n").getBytes());
        out.flush();
        s.close();
    }
    
}

So how does one use this from Ant? Easy !

<target name="jetty-cycle">               
    <parallel>
        <java classname="mypackage.Start" classpathref="test.classpath" fork="true"/>
        <sequential>
            <waitfor>
                <socket server="127.0.0.1" port="8080"/>
            </waitfor>                
            <antcall target="my-tests"/>                            
            <java classname="mypackage.Stop" classpathref="test.classpath"/>                 
        </sequential>
    </parallel>
</target> 

Full disclosure: I adapted the approach from the Jetty code you can find over here ;)

Seam / JSF vs Wicket: performance comparison

A while after Seam support for Apache Wicket was announced, I downloaded Seam and took a look at the Wicket example. Then an idea struck – how about doing a performance comparison – I mean, here was the very same application implemented in JSF and Wicket – right? So I decided to write a JMeter script for both the JSF and Wicket versions of the Seam “hotel booking” example and compare results. I started right away but very soon got tired of waiting for app-server re-starts on my trusty laptop (Jetty has really spoiled me) – and it kind of bothered me that the Seam examples were not in Maven 2 layout. Anyway, at some point I decided to re-write an “EJB-free” version of the Seam booking example using just Wicket and JPA. And use Maven and Jetty. And… one thing led to another – and what I ended up doing is this:

• Adapted the Seam JPA example (the one that does not use EJB) as the baseline application. Converted it into a Maven 2 WAR project which can run on Jetty.
• Implemented what I hope is the exact equivalent of the above using only Wicket and JPA, also Maven-ized and Jetty-fied. Decided to also experiment with some of the ideas in this blog post.
• Wrote a JMeter script for both applications taking care to exercise identical functionality.
• Used an Ant script to run the JMeter scripts in batch mode (passing number of concurrent users as a parameter) and start / stop Jetty in sync.
• Automated the entire cycle of running the load test for 1, 5, 10, 15 and 20 concurrent users including some code to parse the JMeter logs and generate a CSV file of tabular results.
• Also included an Ant target that takes a JVM heap dump at the end of the load test – just before the users are logged out and the HTTP sessions killed.

I was able to re-use the official Seam Wicket example to some extent, mainly the HTML files – but the Java side is almost completely re-written. The Seam booking example covers quite a bit of ground from the framework comparison point of view – for example:

• Security – some pages are secured and redirect to a login page
• Templating – some pages inherit from a common layout with a header / footer defined
• Ajax –
  • hotel search results refresh as you type and a “busy” image is shown during the Ajax request
  • form field validations occur as soon as the field loses focus
• Session scope – some state is stored and retrieved in the session
• Page navigation / state –
  • user navigation state transparently managed by the framework
  • user should be able to work in multiple concurrent conversations (browser tabs)
  • the browser back button should work as expected
• JPA – getting access to the entity manager and transaction management
• Forms – binding, validation and displaying error / info feedback to the user
• Hibernate Validator – annotations on the JPA entity classes are re-used for form-validation

Both sides use the exact same entities, persistence.xml and initial HSQLDB import. I’m using the latest Seam 2.1.1-GA and Wicket 1.3.5. Disclaimer: my Wicket code may not be ideal, and I’ve also experimented with a custom RequestCycle for JPA and tried to use inherited models as far as possible. Instructions on how you can download the source and run the scripts on your local machine are at the end of this blog post.

I may do some follow up blog posts on how the code compares between the Seam / JSF and Wicket implementations, and also share some tips on writing JMeter scripts and automation using Jetty, Ant etc. I’m also expecting to have to make corrections and changes to the code based on feedback. For now, I’ll summarize my observations on performance and memory usage.

Performance:
In the JMeter script, except for the login and logout (first two rows and last row), the actions are executed in a loop ten times for each concurrent user. So if the number of concurrent users is 20, the login and logout actions happen 20 times and the rest 20 x 10 times. The numbers below are average page response time in milliseconds.

Wicket appears to be faster by a wide margin. For two pages (“ajax post search” and “post confirm booking”) the results are a bit closer. This can possibly be explained by the fact that these particular actions display the results from a relatively expensive database query. My amateur profiling attempts suggest that the database query is taking most of the time here.

One thing I have to mention: the “cc number” and “cc name” requests are simulations of the Ajax validations of the credit card number and name fields on the booking form. For these particular requests on the JSF side, the entire form is being POST-ed instead of just the value of the form field being validated when the user tabs out (onblur). So the difference here is quite dramatic. I did try adding ajaxSingle=”true” in the JSF view but it did not appear to work (I used HttpFox while building the JMeter scripts). I can re-post the updated results if someone lets me know what changes need to be made to “book.xhtml” to get the Ajax validation to work as expected.

Memory Usage:
The JMeter script can be told to skip the logout page and I wired up one of the Ant targets to take a JVM heap dump / snapshot as soon as the JMeter script completes. So I can compare what the heap looks like just after a load test when all the concurrent HTTP sessions are alive.

I’m very much a NetBeans user but I have to say that the Eclipse Memory Analyzer is far better than what the built-in NetBeans Profiler offers for looking at JVM heap dumps. Here are some side-by-side screenshots of the heap analysis after running the JMeter script for 20 concurrent users.

“Top Consumers” report below showing the classes that dominate memory usage:

The “dominator tree” report below is very useful to see which objects hold on to the most memory directly as well as indirectly. The column headings after “Class name” are Shallow Heap, Retained Heap and Percentage. Shallow Heap means the memory consumed by a single object and Retained Heap is the sum of shallow sizes of all objects that will be garbage collected if the given object is garbage collected.

Looking at the above two reports we can infer that on the Seam / JSF side, the 20 sessions each take up about 800 KB adding up to around 16 MB total. On the Wicket side the 20 sessions add up to around 1.5 MB. On the Wicket side it is the DiskPageStore that appears to hold the most memory and we can see what is going on here after drilling down a little:

The Wicket DiskPageStore uses SoftReference-s to serialized pages so the memory will be reclaimed by the JVM if needed. And the SerializedPageWithSession holds a WeakReference to the actual page instance (MainPage). You can also spot the byte-array which is the result of page serialization. If a serialized page is requested (perhaps the user hit the browser back-button) and the page is no longer in memory because the SoftReference has been GC-ed – it will be restored from the temp file that the DiskPageStore has been saving pages to.

I’m totally impressed by the Eclipse Memory Analyzer. Here we can see the break-up of the contents of the largest HTTP session on both sides. I think we can safely blame JSF for the lion’s share of memory usage on the left:

Finally, a summary of the heap-dump comparison (for 20 users) collated from the various reports:

Instructions:

Prerequisites:

• JDK 1.5 or greater installed
• Apache Ant installed
• JMeter 2.X available unzipped somewhere (better use latest 2.3.2)

Steps:

• Do a Subversion check out of the source from here: http://perfbench.googlecode.com/svn/trunk/perfbench/
• Create a perfbench/build.properties file that points to your JMeter installation. You can look at the comment in perfbench/build.xml for an example.
• Open a command prompt, change directory to perfbench/seam-jpa
• If running for the first time, use the command “ant jmeter-cycle”. The build script would prompt for the number of threads, so enter “1”. It may take time for all the required JAR files to get downloaded. Once you see Jetty start and stop successfully, you should be all set to run the actual benchmark.
• To start the benchmark run “ant jmeter-cycle-full”. This should take 2 – 3 minutes to run a series of tests for 1, 5, 10, 15 and 20 concurrent users. Results will be dumped into perfbench/target. You can look at the *.csv file at the end for the results.
• You can also run a load test which saves a snapshot of the heap dump towards the end by running “ant jmeter-cycle-heapdump”.
• Repeat the previous 3 steps after changing working directory to perfbench/wicket-jpa

Update: perfbench/build.xml starts Jetty with JVM options “-Xms64m -Xmx64m” and you may need to change this if you want to experiment with more concurrent users.

SpringOne 2008 Day 4

Simplifying JavaServerFaces Development with Spring Faces – Jeremy Grelle
I was running quite late, by the time I was in for this session, Jeremy was out of the slides and busy switching between Eclipse code and the demo web-app. He was covering the “Spring centric” JSF integration approach where you use JSF backed by Spring MVC. And you can use Spring WebFlow to control stateful conversations and navigation rules.

I have used Spring WebFlow 1.0 in the distant past and I it certainly looks like the WebFlow 2.0 syntax is much simpler. It is very much a comprehensive DSL for all kinds of things though and I would guess there is a bit of learning curve. I could spot flashes of the upcoming Spring 3.0 EL++ within the on-screen webflow XML samples, e.g. “bookingService.findHotels(searchCriteria)”. Spring WebFlow appears to encourage doing as much as possible declaratively in the config. Jeremy was recommending that for e.g. instead of implementing action listeners in java code (which is nothing but “glue” code) – just do this directly in the flow definition. This is optional of course, and existing action listeners would continue to participate in the JSF life cycle as normal.

There were a couple of hiccups during the demo which Jeremy recovered from, at one point he wondered out loud if the view did not refresh as expected because of JSF quirks – keeping the whole UI component tree in memory etc.

The Spring Faces tags demos were impressive, for e.g. the rich form input validation messages that gracefully degrade. I especially liked the pop-up demo, which must have been Spring / JavaScript / Dojo behind the scenes. If getting pop-up windows to work with stateful flows, JSF and Ajax is made easier by Spring, that is indeed a Good Thing.

Client / Server Application Development using JSON SOA/REST – Kris Zyp
In this session I realized how much I did not know about what’s going on in the world of cutting edge REST / JSON / Ajax / Dojo / Comet etc. And really, aren’t these the buzzwords of the moment!! So I was glad I picked this session although at times I felt that my head would asplode from JavaScript overload. I had to Google a few unfamiliar things during the course of the session and consistently turned up blog posts written by Kris himself, he is evidently at the forefront of getting a lot of this stuff standardized. I include the relevant links below:

• Service Mapping Description (SMD) – like WSDL for JSON / REST but with far less suck
• JsonRestStore – Dojo has a “Data Store” concept that standardizes an interface or set of APIs to read, write and even scroll through data. JsonRestStore is a data store implementation that can automatically use a given RESTful JSON service, provided it conforms to some conventions.
• JSON Referencing – avoids cyclic references by using only id values with a special key to represent referenced objects – and dojo can even lazy load the whole object from the server, think of this as the JavaScript equivalent of Hibernate proxies.
• JSON Schema – like XML schema but without teh suck
• Bayeux – like JMS for the web – or asynchronous messages over HTTP
• REST Channels – like eventing and pub-sub for the web

Spring Cleaning: Tips for Reducing XML in Spring Configuration – Craig Walls
One of the rare sessions at SpringOne coming from an experienced user rather than a Spring committer. The author of “Spring in Action“, Craig has obviously lots of experience with Spring and I did pick up a few tips. I actually attended two of his sessions back to back hoping to be cured of my allergy to annotations :) This first one covered some basics like using attributes instead of <value>, the “p” namespace, bean inheritance, property editors etc – but it appeared to be quite relevant for the audience. For e.g. I was surprised by the show of hands that not many people had seen the transaction proxy factory bean. Looks like everyone is onto namespaces and annotations nowadays.

The before and after effects for simplifying declarative transaction management were quite convincing and even more convincing was the effect of the Acegi to Spring Security makeover. Spring Security has come a long way indeed :) Craig was pretty emphatic that this is the “best security framework around, period”.

Session ended with how AOP becomes simpler using annotations instead of XML and finally a quick look at JavaConfig. One nice tip I got out of the session is that Jasypt could be used to encrypt externalized config properties like the database connection password.

Spring for the Angle-Bracket Averse: Developing Spring Applications with Absolutely No XML – Craig Walls
That was the last session in SpringOne 2008 (for me) and that’s a long title for a session :) Craig started with a quick look at some alternative contenders like Springy – I had never heard of this (but it is dead now) and the Groovy based Bean Builder. Then he dived into a sample application all done in JavaConfig with zero XML. And no, web.xml doesn’t count. Craig spent time in the IDE, making it clear how this way of configuration could appeal to a Java programmer, you get the benefits of IDE navigation, auto-complete, syntax coloring, type safety, OO, etc. Of course, there are a few annotations to remember and lots of @@@@ symbols in your code. But I guess many people would prefer this to angle brackets and this is just configuration anyway. Session ended with Craig strongly recommending that one should just try JavaConfig once, quite likely you will get hooked. The message was clear that people who used to say Spring sucks because of so much XML just can’t complain anymore.

Update: Solomon has blogged an overall “big picture” summary of SpringOne: http://jroller.com/Solomon/entry/spring_one_themes

Update: InfoQ has a summary report on SpringOne: http://www.infoq.com/news/2008/12/springone-2008

SpringOne 2008 Day 3

RESTful Web Applications with Spring 3.0 – Arjen Poutsma
The “highlight” of the presentation was Arjen’s bright green hair, no really. Rumor has it that he lost a bet? The hall was filled to capacity with people sitting on the sidelines. Some of my take-aways:

• there is no “right way” way to design REST-ful URL-s, some are better than others, subjective. As someone who has used the Flickr API as a reference, that was good for me to hear. REST zealots would argue that the Flickr API is not a good example of “true” RESTful style, whatever that means.
• but using hierarchical URL-s and avoiding query string parameters or method “verbs” in the URL is recommended
• one of the ways you can keep things simple is use the HTTP response codes to determine success / failure of an operation
• use ETags of course, gives you caching advantages, Spring 3.0 can automatically generate the ETag (using an MD5 hash compare)
• expect “representation” support (just like MVC views) for Atom / RSS, JSON etc. XML marshalling support is already there in Spring Web Services
• annotation driven URI templates look quite nice and elegant, can’t wait to try these out

Terracotta – Real Apps, Real Frameworks, Real Use Cases – Ari Zilka (CTO & Founder)
Terracotta sounds too good to be true, I keep wondering what is the catch!? Colin Sampaleanu from SpringSource was also on stage for the first few minutes with a quick intro and comments on how SpringSource and Terracotta are working together to do really big scaling for a client in production. A Terracotta PoC would be high on my to do list when I get back to my workplace.

• they have an oss reference app you can download and try called Examinator on the Spring WebFlow / JPA (Hibernate) stack
• no need for SAN / NAS etc, commodity hardware with local IDE / SCSI etc will do
• Terracotta does all kinds of magic with the JVM behind the scenes. Because it works at the JVM level, you hardly have to make any assumptions when you code, for example your objects don’t even need to be Serializable
• need a cluster wide singleton? no problem
• lots and lots of optimizations like using only deltas, lazy loading, minimizing disk flushes by consolidating fine-grained operations, use local memory for reads as far as possible etc
• use sticky sessions in general as far as possible. and if you use Terracotta with sticky sessions, because of the above mentioned optimizations – Terracotta “will fly”
• I was quite fascinated by Ari’s detailed answer to a question on failover – heartbeating between nodes, nodes hold “elections” to decide who becomes master / passive when new nodes appear or disappear…
• some stats: client they and SpringSource are working with on can hit 20,000 TPS on 16 DELL blades with 5ms response time. Now they are on the way to hitting 50K TPS with the same infrastructure.
• demo of the Terracotta admin console was impressive, he ran a JMeter script. The console can even have you drill down to a thread dump on a node you choose
• made an interesting comment about a client using Terracotta for messaging “across political boundaries”, you can do things you normally expect messaging / JMS to make possible
• in case native stuff is involved, you would have to have a proper design to keep all the ‘state’ you need within the Java boundary as far as possible. mentioned a case where they could cluster even comet sockets this way

Spring and Java EE 6 – Jürgen Höller
This session was quite useful for me because I have not been really bothered about the evolution of Java EE much (true Spring fan :P) but here was a great summary from someone working hard to keep Spring complementing Java EE as far as possible.

• Java EE profiles mean that container vendors don’t need to support all the legacy stuff like EJB 2.0 anymore. I can very well see Spring “tc” server being one of the first containers to be certified when the spec goes final ;)
• Comet support will be in the Servlet 3.0 spec, I didn’t really know that
• Bean Validation is “long overdue” as per Jürgen, this is nice and DRY, your persistence annotations would drive your form binding validations. How soon can the Spring Javascript team get this to automatically generate Ajaxy Dojo validations – that is what I’m wondering :|
• Again and again the message is that the upcoming spec overlaps with existing Spring functionality – e.g. JSF 2.0 @ManagedBean, @ManagedProperty is nothing but Spring @Component, @Value
• Another example, web beans binding annotations are the same thing as Spring @Qualifier
• Jürgen’s opinion is that the Web Beans spec is changing a lot, making a lot of “sharp turns”. Feels that there is a lack of consensus on some decisions like sub-grouping the many annotations into logical packages. Pretty much delayed.
• Says that Spring 2.5 provides a lot of Web Beans functionality already and spring 3.0 closes the gap
• Another comment he made was that the committee has a tough job – there is already a well established component model which is EJB3. And now Web Beans is supposed to do *only* things that EJB cannot do.
• So is EJB 3.1 going to end up as a great complement to Web Beans? Jürgen doesn’t appear to think so.
• The fact that a “singleton session bean” is planned in the spec was news to me. You will be able to even annotate methods as single threaded, async etc
• Whatever specs are final / close would be included in Spring 3.0, for example JPA 2.0 quite likely. But you really can’t predict everything at this point

At the end of the day I caught parts of Adrian Colyer’s keynote. One notable announcement was Spring will support Flex and SpringSource would be partnering with Adobe. Adrian proceeded to do a Grails live-coding demo on stage working with Spring Integration as well, looked like the audience was lapping it all up, I had to leave at this point.

Update: video of the skit by the SpringSource folks before Adrian’s keynote is available here:

http://www.springsource.org/node/845

SpringOne 2008 Day 2

The highlight of the day must surely have been Graeme “Grails” Rocher building a basic Twitter app onstage in 40 min. I missed the fun, chose to go to the “What’s new in Spring 3.0” session by Juergen Hoeller instead.

I won’t repeat stuff verbatim as the slides would be released sometime, but will just jot down my personal impressions from Juergen’s session:

• Starts with a revision of Spring 2.5 config using annotations. Pretty clear that they now recommend the annotation way (even @Autowired !) and minimize the XML
• After yesterdays keynote preview I was wondering if EL for Spring would make sense but it does, I am convinced.
• For example the Spring EL will simplify cases like injecting primitive values (think of a “cacheSize” to be set conditionally). I have a feeling that stuff I am currently doing using a FactoryBean can be avoided with the EL. I am yet to try JavaConfig though.
• Spring will ship with a home grown EL processor, apparently inspired by the JSF “Unified” EL. So it uses #{} kind of syntax. But Spring EL will be more powerful, can even do method invocations. I have not been following the EL scene closely but I distinctly recall a certain other framework that has this feature implemented already. Who came up with this idea first? Hmm.
• The EL support will be tightly integrated with Spring “scopes”. Think conversation, session, application – even exotic things like Spring Batch “step” scope. Sounds elegant.
• Spring will move to an Ivy based custom build and release system in future. Juergen did not say this explicitly but it sounded like they are ditching Maven becuase they want better control over dependencies metadata etc. Juergen said that they would end up using stuff that was / is being tried out in the current Spring WebFlow build system. I had noticed when I tried one of the recent SWF beta-s or RC-s that the build was based on Ivy. If this is true, Maven guys won’t be pleased ;) Anyone care to comment?
• REST support looks neat I could tell a lot of folks were impressed. @PathVariable ftw!
• Juergen promises “conversation scope” is coming that solves the “concurrent windows in same browser” problem. Uh oh, sounds like I have heard that elsewhere as well. Do I smell another flame war on the ServerSide soon? *glee*. The slide says matter-of-factly that this feature is inspired by MyFaces “Orchestra”.
• Next slide brings up “serializability” of session and conversation objects. Solution is proxies that reobtain references on deserialization. Now where have I seen this before? Hmm ;)
• Spring “inheritance based” controllers will still exist but be marked as deprecated in 3.0 in favor of the annotations. I will miss you SimpleFormController and friends, you have served me well!!
• But I won’t miss things like the wonderfully named JUnit 3.X support classes like “AbstractTransactionalDataSourceSpringContextTests”. I see myself using annotations for the Spring JUnit 4 support very soon. As someone that’s not particularly fond of annotations, now that’s saying something.
• Juergen says that his view of Java EE 6 spec progress is that it is delayed, was supposed to be done by end 2008. Word is that they are shooting for JavaOne. Juergen feels that it likely won’t be done even then (June 2009). Hurray for standards. Yay. But some specs have come a long way e.g. JSF 2.0 and JPA 2.0, they may go into Spring 3.0, rest goes into 3.1, 3.2 etc when available.

After that I attended the LinkedIn case study. Interesting, they did a lot of work to customize finer details of how Spring does DI and life cycle stuff. I was not particularly convinced that they needed to go as far as they did – but the message was effectively conveyed that Spring is so easy to extend and bend around to do stuff that you want. For me personally, I had never ever ventured into creating custom Spring XML namespaces etc. and this session gave me some idea as to how to go about it.

After lunch came Jeremy Grelle’s Spring MVC and Spring JavaScript presentation. I had seen most of the slides already, downloaded from one of his talks elsewhere earlier but I did find the demos interesting. Personally I hate mixing JavaScript in your JSP but have to admit that Spring makes things easier than it would be otherwise, typical Spring. This stuff is for you if you are using JSP especially with Tiles – you can do partial refresh of fragments. But the Tiles config looks a little too complex for my liking. I guess I should blame Tiles, not Spring, heh. Jeremy appears to be great fan of Dojo, plugs the session by Dojo member(s) tomorrow. Spring Javascript is currently a wrapper over Dojo, they plan to add other implementations in the future like jQuery, YUI. Sounds a liitle ambitious to me. Spring JS 3.0 will have JSON support and “deep integration with dojo.data”, not sure what that means.

I attended the Morgan Stanley case study at the end which I felt was a “fair and balanced” look at real-life Spring adoption, even having a few slides on what the dev team perceived as Spring limitations – although most points seemed somewhat contrived. Having attended the LinkedIn session earlier it was good to see MS take a somewhat different approach to some of the stuff LI did. Very interesting to see that their architecture was all about message processing (CXF), no app servers, no Tomcat. Instead they bolt a custom TCP channel onto a Spring context. They invested in a Spring subscription and had very positive things to say about it, training, accessibility of SpringSource team etc. Overall I got some nice insights on how to pitch Spring to prospective clients especially those having a lot of legacy code. Very, very nice to see Morgan Stanley talk about killing their in-house frameworks and consolidate on stuff like Spring JDBC. They damn well got it right. Some of the clients (and pointy haired bosses) I have dealt with in the past (and wanted to knock some sense into) come to mind. Yeah if you are reading this blog, you know who you are!!

Here is my twitter feed if you are interested.

SpringOne 2008 Day 1

Now off to bed after the SpringOne kick-off and keynote. My, the venue is grand. Bag and goodies are nice but apparently no Rod Johnson bobblehead this time ;)

Quote as I remember it from Rod Johnson’s keynote: “No one uses Tomcat because they played golf with someone”. Hah! Take that all ye bloated app server vendors ;)

My personal (irreverent) thoughts from the keynote, just a few things I remember:

• Theme this year: “Weapons for the War on Java Complexity”. On the swag T-shirt as well.
• Yep, more stats and proof that Tomcat is the dominant app-server and even the other app-server deployments mostly run Spring apps. Spring world domination complete.
• A lot on Spring WebFlow, looks that it plays a big role in the overall roadmap. Now did he actually say that he was not completely convinced that Spring WebFlow was simple enough and made complete sense until recently? Hmm. Yes, SWF syntax much simpler in 2.0 compared to 1.0. Rod Johnson stresses that SWF makes sense for guided page navigation, back button problem, opening multiple browser windows at the same time etc. Spring JavaScript
• Talks about the Big Choice JSF vs non-JSF, compares this to Republicans vs Democrats, jokes about how ServerSide threads on JSF get hijacked by people who assert that JSF sucks. Rod does not say what he *really* feels about JSF. I will buy a beer for anyone who can tell ;)
• Hardly any XML shown in the slides, mostly annotations. Guess I can’t avoid this stuff any more
• In fact “admits” that the old Spring MVC was not really the “right way”. The inheritance based controllers will be deprecated in Spring 3.0. What the! Breaking news: Rod Johnson says that concrete inheritance “sucks”.
• DM server, AMS, Oracle extensions. Lot’s of “SpringSource” stuff.
• Aha, something new unveiled, SpringSource “tc” – Tomcat with management and enterprise features added
• I can buy into this more than the OSGi stuff. Rod says that barrier to Tomcat usage in production is the management and enterprise / deployment features. Hmmm, probably. So you take your existing WAR and it should run on this. Not yet clear if this uses the DM server under the hood, think so.
• Spring 3.0 will introduce REST support and some kind of an Expression Language. EL that you can embed in annotations and stuff huh!? Now that is interesting, reminds me of something I’ve seen elsewhere. Need to catch the first session tomorrow on Spring 3.0 by Juergen.

Wicket and GWT compared with code

I have been using Wicket for a while now and I’ve occasionally wondered if GWT provides any kind of advantage over Wicket. Recently I got a chance to do a comparison and instead of coding a simple “Hello World” kind of example, I decided to try something a little more complex. Hopefully this brings out the differences between Wicket and GWT more clearly.

Functionality

The target functionality is a one-page application that shows data in a single table. Item counts are displayed categorized under multiple “spaces” (workspaces). There is a summary row. The user can click on a space to expand the count of items grouped by status. The screenshots below show how the table looks when fully collapsed and then when one or more spaces are expanded.

It may look simple, but some of the tricky parts are:

• The number and names of the possible status types can be different for different spaces. For example, the first space above has 3 status types and the second 2. This means that we can’t use different columns to group this data. Multiple rows are used instead to display the break-up when expanded.
• Some of the table cells have to be merged to represent the grouping by space and then by status. In HTML terms, this requires some careful control of TD “colspan” and “rowspan” attributes.
• In addition to the cell merging described above, the style / color of the different cells has to be controlled to differentiate the total count from that grouped by status. For example, the grand total on the last row is in bold. In HTML terms, this requires control over the CSS “class” attribute, coordinated with a style-sheet.

First, let us look at the Java code common to the GWT and Wicket implementations.

The “Space” Domain Object

The class above represents a “space” and it internally uses a Map to hold status names and corresponding counts. There is a method that calculates the total count, a getter for the space name and list of states, and helper methods to add and get status data.

DashrService.java

As the functionality is kind of a ‘dashboard’ view – and inspired by the “Cheesr” example that the Wicket In Action book uses, I decided to call this application “Dashr”. “DashrService.java” is a singleton that returns a list of spaces with dummy data. The relevant code is shown below:

So on to the comparison then. In the side-by-side layouts below, GWT is on the left and Wicket is on the right.

Project Structure

Although it is not really a domain class, “DashrService.java” has been kept in the “domain” package along with “Space.java” to keep things simple.

On the GWT side, the recommended package structure and naming conventions (“client”, “public”, etc.) are being used. Our example is a pure client-side application and server-side communication or RPC is not being used at all. All the client-side code and behavior is in “DashrEntryPoint.java” itself. “Dashr.html” simply serves as a placeholder for the rendered application.

It may first appear that Wicket requires more files than GWT but one difference is that the GWT code is not communicating with any server-side functionality at all. Wicket would feel more natural to Java web-app developers familiar with web.xml and the like, and integrating additional server side code or frameworks such as Spring or Hibernate would be much more straightforward.

Another thing is that the Wicket application has been modularized into multiple Java components and corresponding HTML files. It was not really possible to do something similar for the GWT application which has all the UI code in a single file. The reasons for this difference will become clear when we look at the code in detail.

Framework Configuration

GWT needs to be told where to find the Java classes that have to be converted into JavaScript. If this path is not explicitly set using an XML element called “source” it defaults to “client” – which is the GWT recommended convention. Our main module config file “Dashr.gwt.xml” does not have the source path declared. So this means that all the Java code in the directory “dashr/gwt/client” (and any sub-directories) will be processed by the GWT Java to JavaScript compiler.

Since “Space.java” and “DashrService.java” are dependencies for our GWT based UI code, we need to ensure that they are included in the Java to JavaScript conversion as well. So in addition to the “User” core framework module that all GWT apps must inherit, we have a reference to a custom module – “dashr.domain.DashrDomain”. “DashrDomain.gwt.xml” declares the source path to be an empty string which in relative-path terms works out to be the same folder the XML file is in – and this is how the “dashr/domain” directory is included for our example. Those who intend to use GWT to connect to server-side Java code would need to do something like this – i.e. include all dependencies needed by UI code, typically domain objects that happen to already exist – like in our example.

The UI class that acts as the “entry point” has to be configured for a GWT application. In our example we have only one UI class and this itself will be the entry point.

One more thing the main GWT config does is declare “Dashr.css” as the CSS file for our application. A GWT convention-over-configuration rule applies here as well which is that resources like CSS files and images are looked for in the “public” folder relative to the XML file by default.

For Wicket, there is no XML configuration but an “application” class that can hold configuration in Java code. The web.xml file is not really Wicket configuration but it is shown above for completeness and it names the Wicket “application” class used by the framework servlet / filter. In our example, “DashrApplication.java” is just pointing to the designated “home page”. Our home page is “DashrPage.java” for which the HTML markup is in “DashrPage.html”. The Wicket “home page” can be considered equivalent to the GWT “entry point”.

UI Implementation

Because of the differences between GWT and Wicket – the code below does not really line up side by side. To make it easier to see how the code compares – some key sections from the GWT side have been mapped to the Wicket side using blue arrows. I have not mapped every possible thing because then the picture would be out of control! You can click on the picture to view a bigger version

Observations:

1. Wicket gives you complete control over the HTML. To render a table we use a “ListView” repeater control [DashrPage.java:18] which manipulates a <TR> section. In the case of GWT however, we have to use an abstraction over an HTML table either HTMLTable or FlexTable. Since we require control over the table “colspan” and “rowspan” attributes, we have to use the GWT “FlexTable” as the basic “HTMLTable” does not support this. What I experienced is that having to use the GWT API to output HTML limits you in certain ways, for example on the Wicket side I was able to freely include <TH> cells with <TD> cells but within the GWT table I could not mix <TH> cells. This may explain why the GWT “Dashr.css” file has an extra CSS class called “header” defined to compensate.
2. Using Wicket’s built-in Ajax support, getting the expand / collapse functionality to work was very easy. It was a simple matter of replacing one (or more) rows of the HTML table over Ajax [RowCollapsedPanel.java:19, RowExpandedPanel:29]. Trying to do the same thing using GWT turned out to be quite a puzzle-solving exercise. To dynamically add or remove rows on a GWT table you have to know the row index and in our case the number of rows in the table changes dynamically. I ended up using a HashMap [DashrEntryPoint.java:21] to track the table row for each Space. This explains the mysterious code you see at lines 98 and 124 in DashrEntryPoint.java to update the Map when the table is changed.
3. To dynamically set the contents of a table cell in GWT, you have to explicitly get a reference to the cell using the row-index and column-index. But in Wicket it is very easy to add or replace text as long as you know the wicket:id of the HTML element and it works fine for <TD> as well.
4. One of the biggest problems I found with GWT is the poor separation of concerns. You can see a lot of code in DashrEntryPoint.java doing CSS styling on the HTML table and cells, for e.g. the calls to getRowFormatter() and getCellFormatter(). GWT does allow you to use a standard CSS file but you have to programmatically set the styles on your widgets in Java code. You can see for example how lines 42-43 in DashrEntryPoint.java map cleanly to markup and code on the Wicket side. The Wicket implementation does not have any CSS mixed in the Java code at all – it is all in the HTML where it belongs, and where web-designers can work freely. Most GWT widgets make use of pre-defined CSS names so this may not be a big problem for simple applications, but the web-designer would still need to know which CSS class names to use.
5. This example only has a single widget on one page, but when you place multiple widgets on a screen, you have to think in terms of layout managers just like Swing. I have not gone into what GWT provides in detail but it looks suspiciously like the days of struggling with GridBagLayout :) I prefer the much simpler and effective Wicket approach of managing layout using standard HTML and I personally feel that web-designers need to be able to work directly with the HTML (and CSS) and have better control over things like cross-browser rendering quirks.
6. On the GWT side Dashr.html is just a placeholder for all the JavaScript stuff that will happen at run-time. So when testing or debugging, if you do “view source” you see nothing, just the skeletal Dashr.html! This was a real pain when I was trying to tweak things like the CSS styling by trial and error. Especially when you are using hosted mode you can say goodbye to things like the web developer FireFox extension. When it comes to Wicket, you are dealing with a “normal” web-app and being able to look at the rendered HTML or DOM at any time is something you take for granted.
7. One of Wicket’s strong points is that you can take chunks of HTML and model them as reusable panels. This makes the code modular and easier to read and it is obvious what the intent of “RowCollapsedPanel” and “RowExpandedPanel” is. Maybe with some more thinking and effort on the GWT side I could have had the inner classes ExpandClickListener and CollapseClickListener in separate files. But the code in the “onClick” event handlers is tightly coupled to the “table” and “spaceRowMap” properties of the enclosing class. I considered passing these as parameters in the constructor but decided it wasn’t worth the extra complexity and that the readability of the GWT code would reduce even more.
8. The big difference between GWT and Wicket is of course that at run-time, all the GWT code becomes JavaScript and executes on the client browser. This may require the developer to think a little more about the implications of certain design choices. For example on lines 51 and 83 of DashrEntryPoint.java, a new instance of a ClickListener is being instantiated for each Space. On the server side, I wouldn’t think twice about dynamically instantiating many objects to render a page but in GWT I would have to start thinking about the impact this would have especially when everything ultimately becomes client-side JavaScript. The GWT tutorial also has some recommendations regarding this.
9. The GWT host HTML page Dashr.html contains a hidden IFRAME to support use of the browser back-button. To be able to use this feature properly requires some extra effort as detailed in this page of the GWT documentation. I did not spend time trying to understand this and I simply used null for the “state token” on lines 50 and 82 of DashrEntryPoint.java. The Wicket application transparently supports the browser back-button. If you really need bookmarkable URLs, you then need to do some extra work in Wicket just like you have to do in GWT.
10. The conversion to JavaScript takes more time than normal Java compilation. I found a mention of a real-life project experiencing GWT compile times of up to 18 minutes.
11. On a more positive note, I used the latest GWT 1.5 and whatever little Java 5 features I used (generics, enhanced for loop) worked fine. You still need to be aware of some gotchas though.
12. The GWT code does not communicate with the server at all which can indeed be considered one of the advantages of GWT because of performance. But it depends. What if I really wanted the data to come from the server? Then I would need to understand how to make server calls and perhaps even GWT serialization. The “Dashr” requirement is not something I cooked up and I am using something similar in an actual project. In the real-life requirement when a Space is expanded, the data has to come from the server (over Ajax) because it is an expensive “count (*)” kind of query in the database. In other words, the expanded data for all spaces cannot be loaded upfront in one go. Handling this kind of stuff in Wicket comes naturally, you *are* on the server all the time. And if you use Hibernate you have to be aware of certain issues with GWT. I hope to expand this example with RPC into another blog post soon and also explain why I think concepts like SOFEA have drawbacks that need to be taken into consideration.
13. A few other points worth noting that are not addressed in the example, I personally find GWT’s approach to JavaScript integration, unit testing and especially internationalization more complicated than Wicket.

Overall, a perception of GWT I couldn’t get rid of while working with it is that it feels like a hack – but of course, a very well engineered hack. Maybe if you really have a lot of stuff you want to do on only the client side, it is fine. From the perspective of someone used to Java server-side programming, there are too many things about the unique approach that leak through – not being able to do “view-source” and the added complexity of RPC being some of them. And for example here below is a screen-shot of what I get when I “compile” the GWT application into JavaScript:

It is quite weird to see a one-page application result in a bunch of files like this. There is one JavaScript file but even the HTML files contain almost nothing but JavaScript. All the JavaScript is mini-fied and heavily obfuscated. I guess one can get used to it :P

You can’t do this using Adobe Flex!

I initially had an ambitious plan to include Adobe Flex and make this a three-way comparison. But as far as I can tell, there is simply no way to get the exact table grid behavior we require using Flex! Simple things like “rowspan” are simply out of the question. Some more recent Flex widgets like the Advanced Data Grid support grouping of data and expanding grouped items etc. – but come nowhere close to the requirement outlined at the beginning of this post.

I may have missed something, so do let me know in the comments.

Related previous blog post: A Wicket User Tries JSF – comparison of Wicket and JSF using a simpler example.