SpringOne 2008 Day 2

The highlight of the day must surely have been Graeme “Grails” Rocher building a basic Twitter app onstage in 40 min. I missed the fun, chose to go to the “What’s new in Spring 3.0″ session by Juergen Hoeller instead.

I won’t repeat stuff verbatim as the slides would be released sometime, but will just jot down my personal impressions from Juergen’s session:

  • Starts with a revision of Spring 2.5 config using annotations. Pretty clear that they now recommend the annotation way (even @Autowired !) and minimize the XML
  • After yesterdays keynote preview I was wondering if EL for Spring would make sense but it does, I am convinced.
  • For example the Spring EL will simplify cases like injecting primitive values (think of a “cacheSize” to be set conditionally). I have a feeling that stuff I am currently doing using a FactoryBean can be avoided with the EL. I am yet to try JavaConfig though.
  • Spring will ship with a home grown EL processor, apparently inspired by the JSF “Unified” EL. So it uses #{} kind of syntax. But Spring EL will be more powerful, can even do method invocations. I have not been following the EL scene closely but I distinctly recall a certain other framework that has this feature implemented already. Who came up with this idea first? Hmm.
  • The EL support will be tightly integrated with Spring “scopes”. Think conversation, session, application – even exotic things like Spring Batch “step” scope. Sounds elegant.
  • Spring will move to an Ivy based custom build and release system in future. Juergen did not say this explicitly but it sounded like they are ditching Maven becuase they want better control over dependencies metadata etc. Juergen said that they would end up using stuff that was / is being tried out in the current Spring WebFlow build system. I had noticed when I tried one of the recent SWF beta-s or RC-s that the build was based on Ivy. If this is true, Maven guys won’t be pleased ;) Anyone care to comment?
  • REST support looks neat I could tell a lot of folks were impressed. @PathVariable ftw!
  • Juergen promises “conversation scope” is coming that solves the “concurrent windows in same browser” problem. Uh oh, sounds like I have heard that elsewhere as well. Do I smell another flame war on the ServerSide soon? *glee*. The slide says matter-of-factly that this feature is inspired by MyFaces “Orchestra”.
  • Next slide brings up “serializability” of session and conversation objects. Solution is proxies that reobtain references on deserialization. Now where have I seen this before? Hmm ;)
  • Spring “inheritance based” controllers will still exist but be marked as deprecated in 3.0 in favor of the annotations. I will miss you SimpleFormController and friends, you have served me well!!
  • But I won’t miss things like the wonderfully named JUnit 3.X support classes like “AbstractTransactionalDataSourceSpringContextTests”. I see myself using annotations for the Spring JUnit 4 support very soon. As someone that’s not particularly fond of annotations, now that’s saying something.
  • Juergen says that his view of Java EE 6 spec progress is that it is delayed, was supposed to be done by end 2008. Word is that they are shooting for JavaOne. Juergen feels that it likely won’t be done even then (June 2009). Hurray for standards. Yay. But some specs have come a long way e.g. JSF 2.0 and JPA 2.0, they may go into Spring 3.0, rest goes into 3.1, 3.2 etc when available.

After that I attended the LinkedIn case study. Interesting, they did a lot of work to customize finer details of how Spring does DI and life cycle stuff. I was not particularly convinced that they needed to go as far as they did – but the message was effectively conveyed that Spring is so easy to extend and bend around to do stuff that you want. For me personally, I had never ever ventured into creating custom Spring XML namespaces etc. and this session gave me some idea as to how to go about it.

After lunch came Jeremy Grelle’s Spring MVC and Spring JavaScript presentation. I had seen most of the slides already, downloaded from one of his talks elsewhere earlier but I did find the demos interesting. Personally I hate mixing JavaScript in your JSP but have to admit that Spring makes things easier than it would be otherwise, typical Spring. This stuff is for you if you are using JSP especially with Tiles – you can do partial refresh of fragments. But the Tiles config looks a little too complex for my liking. I guess I should blame Tiles, not Spring, heh. Jeremy appears to be great fan of Dojo, plugs the session by Dojo member(s) tomorrow. Spring Javascript is currently a wrapper over Dojo, they plan to add other implementations in the future like jQuery, YUI. Sounds a liitle ambitious to me. Spring JS 3.0 will have JSON support and “deep integration with dojo.data”, not sure what that means.

I attended the Morgan Stanley case study at the end which I felt was a “fair and balanced” look at real-life Spring adoption, even having a few slides on what the dev team perceived as Spring limitations – although most points seemed somewhat contrived. Having attended the LinkedIn session earlier it was good to see MS take a somewhat different approach to some of the stuff LI did. Very interesting to see that their architecture was all about message processing (CXF), no app servers, no Tomcat. Instead they bolt a custom TCP channel onto a Spring context. They invested in a Spring subscription and had very positive things to say about it, training, accessibility of SpringSource team etc. Overall I got some nice insights on how to pitch Spring to prospective clients especially those having a lot of legacy code. Very, very nice to see Morgan Stanley talk about killing their in-house frameworks and consolidate on stuff like Spring JDBC. They damn well got it right. Some of the clients (and pointy haired bosses) I have dealt with in the past (and wanted to knock some sense into) come to mind. Yeah if you are reading this blog, you know who you are!!

Here is my twitter feed if you are interested.

SpringOne 2008 Day 1

Now off to bed after the SpringOne kick-off and keynote. My, the venue is grand. Bag and goodies are nice but apparently no Rod Johnson bobblehead this time ;)

Quote as I remember it from Rod Johnson’s keynote: “No one uses Tomcat because they played golf with someone”. Hah! Take that all ye bloated app server vendors ;)

My personal (irreverent) thoughts from the keynote, just a few things I remember:

  • Theme this year: “Weapons for the War on Java Complexity”. On the swag T-shirt as well.
  • Yep, more stats and proof that Tomcat is the dominant app-server and even the other app-server deployments mostly run Spring apps. Spring world domination complete.
  • A lot on Spring WebFlow, looks that it plays a big role in the overall roadmap. Now did he actually say that he was not completely convinced that Spring WebFlow was simple enough and made complete sense until recently? Hmm. Yes, SWF syntax much simpler in 2.0 compared to 1.0. Rod Johnson stresses that SWF makes sense for guided page navigation, back button problem, opening multiple browser windows at the same time etc. Spring JavaScript
  • Talks about the Big Choice JSF vs non-JSF, compares this to Republicans vs Democrats, jokes about how ServerSide threads on JSF get hijacked by people who assert that JSF sucks. Rod does not say what he *really* feels about JSF. I will buy a beer for anyone who can tell ;)
  • Hardly any XML shown in the slides, mostly annotations. Guess I can’t avoid this stuff any more
  • In fact “admits” that the old Spring MVC was not really the “right way”. The inheritance based controllers will be deprecated in Spring 3.0. What the! Breaking news: Rod Johnson says that concrete inheritance “sucks”.
  • DM server, AMS, Oracle extensions. Lot’s of “SpringSource” stuff.
  • Aha, something new unveiled, SpringSource “tc” – Tomcat with management and enterprise features added
  • I can buy into this more than the OSGi stuff. Rod says that barrier to Tomcat usage in production is the management and enterprise / deployment features. Hmmm, probably. So you take your existing WAR and it should run on this. Not yet clear if this uses the DM server under the hood, think so.
  • Spring 3.0 will introduce REST support and some kind of an Expression Language. EL that you can embed in annotations and stuff huh!? Now that is interesting, reminds me of something I’ve seen elsewhere. Need to catch the first session tomorrow on Spring 3.0 by Juergen.

Wicket and GWT compared with code

I have been using Wicket for a while now and I’ve occasionally wondered if GWT provides any kind of advantage over Wicket. Recently I got a chance to do a comparison and instead of coding a simple “Hello World” kind of example, I decided to try something a little more complex. Hopefully this brings out the differences between Wicket and GWT more clearly.

Functionality

The target functionality is a one-page application that shows data in a single table. Item counts are displayed categorized under multiple “spaces” (workspaces). There is a summary row. The user can click on a space to expand the count of items grouped by status. The screenshots below show how the table looks when fully collapsed and then when one or more spaces are expanded.

It may look simple, but some of the tricky parts are:

  • The number and names of the possible status types can be different for different spaces. For example, the first space above has 3 status types and the second 2. This means that we can’t use different columns to group this data. Multiple rows are used instead to display the break-up when expanded.
  • Some of the table cells have to be merged to represent the grouping by space and then by status. In HTML terms, this requires some careful control of TD “colspan” and “rowspan” attributes.
  • In addition to the cell merging described above, the style / color of the different cells has to be controlled to differentiate the total count from that grouped by status. For example, the grand total on the last row is in bold. In HTML terms, this requires control over the CSS “class” attribute, coordinated with a style-sheet.

First, let us look at the Java code common to the GWT and Wicket implementations.

The “Space” Domain Object

The class above represents a “space” and it internally uses a Map to hold status names and corresponding counts. There is a method that calculates the total count, a getter for the space name and list of states, and helper methods to add and get status data.

DashrService.java

As the functionality is kind of a ‘dashboard’ view – and inspired by the “Cheesr” example that the Wicket In Action book uses, I decided to call this application “Dashr”. “DashrService.java” is a singleton that returns a list of spaces with dummy data. The relevant code is shown below:

So on to the comparison then. In the side-by-side layouts below, GWT is on the left and Wicket is on the right.

Project Structure

Although it is not really a domain class, “DashrService.java” has been kept in the “domain” package along with “Space.java” to keep things simple.

On the GWT side, the recommended package structure and naming conventions (“client”, “public”, etc.) are being used. Our example is a pure client-side application and server-side communication or RPC is not being used at all. All the client-side code and behavior is in “DashrEntryPoint.java” itself. “Dashr.html” simply serves as a placeholder for the rendered application.

It may first appear that Wicket requires more files than GWT but one difference is that the GWT code is not communicating with any server-side functionality at all. Wicket would feel more natural to Java web-app developers familiar with web.xml and the like, and integrating additional server side code or frameworks such as Spring or Hibernate would be much more straightforward.

Another thing is that the Wicket application has been modularized into multiple Java components and corresponding HTML files. It was not really possible to do something similar for the GWT application which has all the UI code in a single file. The reasons for this difference will become clear when we look at the code in detail.

Framework Configuration

GWT needs to be told where to find the Java classes that have to be converted into JavaScript. If this path is not explicitly set using an XML element called “source” it defaults to “client” – which is the GWT recommended convention. Our main module config file “Dashr.gwt.xml” does not have the source path declared. So this means that all the Java code in the directory “dashr/gwt/client” (and any sub-directories) will be processed by the GWT Java to JavaScript compiler.

Since “Space.java” and “DashrService.java” are dependencies for our GWT based UI code, we need to ensure that they are included in the Java to JavaScript conversion as well. So in addition to the “User” core framework module that all GWT apps must inherit, we have a reference to a custom module – “dashr.domain.DashrDomain”. “DashrDomain.gwt.xml” declares the source path to be an empty string which in relative-path terms works out to be the same folder the XML file is in – and this is how the “dashr/domain” directory is included for our example. Those who intend to use GWT to connect to server-side Java code would need to do something like this – i.e. include all dependencies needed by UI code, typically domain objects that happen to already exist – like in our example.

The UI class that acts as the “entry point” has to be configured for a GWT application. In our example we have only one UI class and this itself will be the entry point.

One more thing the main GWT config does is declare “Dashr.css” as the CSS file for our application. A GWT convention-over-configuration rule applies here as well which is that resources like CSS files and images are looked for in the “public” folder relative to the XML file by default.

For Wicket, there is no XML configuration but an “application” class that can hold configuration in Java code. The web.xml file is not really Wicket configuration but it is shown above for completeness and it names the Wicket “application” class used by the framework servlet / filter. In our example, “DashrApplication.java” is just pointing to the designated “home page”. Our home page is “DashrPage.java” for which the HTML markup is in “DashrPage.html”. The Wicket “home page” can be considered equivalent to the GWT “entry point”.

UI Implementation

Because of the differences between GWT and Wicket – the code below does not really line up side by side. To make it easier to see how the code compares – some key sections from the GWT side have been mapped to the Wicket side using blue arrows. I have not mapped every possible thing because then the picture would be out of control! You can click on the picture to view a bigger version

Observations:

  1. Wicket gives you complete control over the HTML. To render a table we use a “ListView” repeater control [DashrPage.java:18] which manipulates a <TR> section. In the case of GWT however, we have to use an abstraction over an HTML table either HTMLTable or FlexTable. Since we require control over the table “colspan” and “rowspan” attributes, we have to use the GWT “FlexTable” as the basic “HTMLTable” does not support this. What I experienced is that having to use the GWT API to output HTML limits you in certain ways, for example on the Wicket side I was able to freely include <TH> cells with <TD> cells but within the GWT table I could not mix <TH> cells. This may explain why the GWT “Dashr.css” file has an extra CSS class called “header” defined to compensate.
  2. Using Wicket’s built-in Ajax support, getting the expand / collapse functionality to work was very easy. It was a simple matter of replacing one (or more) rows of the HTML table over Ajax [RowCollapsedPanel.java:19, RowExpandedPanel:29]. Trying to do the same thing using GWT turned out to be quite a puzzle-solving exercise. To dynamically add or remove rows on a GWT table you have to know the row index and in our case the number of rows in the table changes dynamically. I ended up using a HashMap [DashrEntryPoint.java:21] to track the table row for each Space. This explains the mysterious code you see at lines 98 and 124 in DashrEntryPoint.java to update the Map when the table is changed.
  3. To dynamically set the contents of a table cell in GWT, you have to explicitly get a reference to the cell using the row-index and column-index. But in Wicket it is very easy to add or replace text as long as you know the wicket:id of the HTML element and it works fine for <TD> as well.
  4. One of the biggest problems I found with GWT is the poor separation of concerns. You can see a lot of code in DashrEntryPoint.java doing CSS styling on the HTML table and cells, for e.g. the calls to getRowFormatter() and getCellFormatter(). GWT does allow you to use a standard CSS file but you have to programmatically set the styles on your widgets in Java code. You can see for example how lines 42-43 in DashrEntryPoint.java map cleanly to markup and code on the Wicket side. The Wicket implementation does not have any CSS mixed in the Java code at all – it is all in the HTML where it belongs, and where web-designers can work freely. Most GWT widgets make use of pre-defined CSS names so this may not be a big problem for simple applications, but the web-designer would still need to know which CSS class names to use.
  5. This example only has a single widget on one page, but when you place multiple widgets on a screen, you have to think in terms of layout managers just like Swing. I have not gone into what GWT provides in detail but it looks suspiciously like the days of struggling with GridBagLayout :) I prefer the much simpler and effective Wicket approach of managing layout using standard HTML and I personally feel that web-designers need to be able to work directly with the HTML (and CSS) and have better control over things like cross-browser rendering quirks.
  6. On the GWT side Dashr.html is just a placeholder for all the JavaScript stuff that will happen at run-time. So when testing or debugging, if you do “view source” you see nothing, just the skeletal Dashr.html! This was a real pain when I was trying to tweak things like the CSS styling by trial and error. Especially when you are using hosted mode you can say goodbye to things like the web developer FireFox extension. When it comes to Wicket, you are dealing with a “normal” web-app and being able to look at the rendered HTML or DOM at any time is something you take for granted.
  7. One of Wicket’s strong points is that you can take chunks of HTML and model them as reusable panels. This makes the code modular and easier to read and it is obvious what the intent of “RowCollapsedPanel” and “RowExpandedPanel” is. Maybe with some more thinking and effort on the GWT side I could have had the inner classes ExpandClickListener and CollapseClickListener in separate files. But the code in the “onClick” event handlers is tightly coupled to the “table” and “spaceRowMap” properties of the enclosing class. I considered passing these as parameters in the constructor but decided it wasn’t worth the extra complexity and that the readability of the GWT code would reduce even more.
  8. The big difference between GWT and Wicket is of course that at run-time, all the GWT code becomes JavaScript and executes on the client browser. This may require the developer to think a little more about the implications of certain design choices. For example on lines 51 and 83 of DashrEntryPoint.java, a new instance of a ClickListener is being instantiated for each Space. On the server side, I wouldn’t think twice about dynamically instantiating many objects to render a page but in GWT I would have to start thinking about the impact this would have especially when everything ultimately becomes client-side JavaScript. The GWT tutorial also has some recommendations regarding this.
  9. The GWT host HTML page Dashr.html contains a hidden IFRAME to support use of the browser back-button. To be able to use this feature properly requires some extra effort as detailed in this page of the GWT documentation. I did not spend time trying to understand this and I simply used null for the “state token” on lines 50 and 82 of DashrEntryPoint.java. The Wicket application transparently supports the browser back-button. If you really need bookmarkable URLs, you then need to do some extra work in Wicket just like you have to do in GWT.
  10. The conversion to JavaScript takes more time than normal Java compilation. I found a mention of a real-life project experiencing GWT compile times of up to 18 minutes.
  11. On a more positive note, I used the latest GWT 1.5 and whatever little Java 5 features I used (generics, enhanced for loop) worked fine. You still need to be aware of some gotchas though.
  12. The GWT code does not communicate with the server at all which can indeed be considered one of the advantages of GWT because of performance. But it depends. What if I really wanted the data to come from the server? Then I would need to understand how to make server calls and perhaps even GWT serialization. The “Dashr” requirement is not something I cooked up and I am using something similar in an actual project. In the real-life requirement when a Space is expanded, the data has to come from the server (over Ajax) because it is an expensive “count (*)” kind of query in the database. In other words, the expanded data for all spaces cannot be loaded upfront in one go. Handling this kind of stuff in Wicket comes naturally, you *are* on the server all the time. And if you use Hibernate you have to be aware of certain issues with GWT. I hope to expand this example with RPC into another blog post soon and also explain why I think concepts like SOFEA have drawbacks that need to be taken into consideration.
  13. A few other points worth noting that are not addressed in the example, I personally find GWT’s approach to JavaScript integration, unit testing and especially internationalization more complicated than Wicket.

Overall, a perception of GWT I couldn’t get rid of while working with it is that it feels like a hack – but of course, a very well engineered hack. Maybe if you really have a lot of stuff you want to do on only the client side, it is fine. From the perspective of someone used to Java server-side programming, there are too many things about the unique approach that leak through – not being able to do “view-source” and the added complexity of RPC being some of them. And for example here below is a screen-shot of what I get when I “compile” the GWT application into JavaScript:

It is quite weird to see a one-page application result in a bunch of files like this. There is one JavaScript file but even the HTML files contain almost nothing but JavaScript. All the JavaScript is mini-fied and heavily obfuscated. I guess one can get used to it :P

You can’t do this using Adobe Flex!

I initially had an ambitious plan to include Adobe Flex and make this a three-way comparison. But as far as I can tell, there is simply no way to get the exact table grid behavior we require using Flex! Simple things like “rowspan” are simply out of the question. Some more recent Flex widgets like the Advanced Data Grid support grouping of data and expanding grouped items etc. – but come nowhere close to the requirement outlined at the beginning of this post.

I may have missed something, so do let me know in the comments.

Related previous blog post: A Wicket User Tries JSF – comparison of Wicket and JSF using a simpler example.

An alternative Maven plugin for Ant and NetBeans

A while back I wrote a custom Maven plugin to “escape” from Maven because I personally prefer Ant for build scripting. I now generally use Maven only for JAR file dependency management and my plugin acts as a bridge between the Maven and Ant worlds. The plugin can automatically generate an Ant build.xml file from the Maven pom.xml file and it supports web-application (WAR) projects. A few people aware of the existence of this ‘underground’ plugin have been asking me for more details, and since I got a chance to tweak things recently, I’m putting down some information in this blog post on how to get the plugin and try it out. Maybe you will find it useful.

Some of the features the plugin provides are:

  • Generated build file takes care of WAR packaging and deploying to Tomcat
  • Once generated, Ant build is independent of Maven and offline
  • Build file is IDE-independent, portable and includes targets for running JUnit tests as well

NetBeans users may find some of the advanced features interesting:

  • Generation of the following different types of NetBeans projects:
    • Java Free Form Project
    • NetBeans ‘native’ Java SE Project
    • NetBeans ‘native’ Java EE (web) project
  • The Ant build (for Free Form project mode) is simple, human-readable and easily customizable instead of the horribly complex “build-impl.xml” that NetBeans users are familiar with
  • In addition to the standard clean, compile, run actions etc., the generated Free Form project supports the following NetBeans-specific IDE-integration when a file is selected in the project explorer window:
    • run single file
    • debug single file
    • run JUnit test for single file
    • run JUnit test in debug mode for single file
    • debug web application
    • Hot Deploy single file into debug session

You just need the light-weight Java SE version of NetBeans in order to start and stop Tomcat, debug and even hot-deploy classes for your web-app. No extra plugins are required for e.g. Maven or Tomcat support. You can try this out for yourself by following the instructions below.

I’ll use the Wicket “quickstart” Maven Archetype as an example of how you can quickly get up and running using Ant on a Maven web-project. You need Maven 2 installed as a pre-requisite. The Wicket “quickstart” can be found here:

http://wicket.apache.org/quickstart.html

Open a command prompt, cut and paste the magic command and you get a simple web-application project along with a Maven POM definition, ideal for testing out the plugin features. This is the command I used for this example:

mvn archetype:create -DarchetypeGroupId=org.apache.wicket -DarchetypeArtifactId=wicket-archetype-quickstart -DarchetypeVersion=1.3.4 -DgroupId=com.mycompany -DartifactId=myproject

This will create a directory called “myproject” which contains a “pom.xml” file as well as some source code. We need to add a few lines to the pom.xml in order to use the custom plugin. First declare the repository for the plugin as follows by adding this snippet just above the <dependencies> section:

<pluginRepositories>
    <pluginRepository>
        <id>jtrac.info</id>
        <url>http://j-trac.sourceforge.net/repository</url>
    </pluginRepository>
</pluginRepositories>

And within the <plugins> section, just add these four lines:

<plugin>
    <groupId>info.jtrac</groupId>
    <artifactId>maven-antprops-plugin</artifactId>
</plugin>

That’s all that needs to be added to the POM, just 10 lines that don’t get in the way of any other Maven stuff you may want to do. To generate the Ant script, change to the newly created project folder and run the following command:

mvn antprops:generate

This will create a build.xml file and a “build-deps.properties” file. The properties file is a Plain Old Java Properties file that contains classpath information extracted from Maven as well as a list of dependencies with “runtime” scope which is used to create the “WEB-INF/lib” part of your WAR.

If you have Tomcat and Ant available on your system you can start Tomcat and deploy the WAR right away from the command line. But first you will need to point the build-file to where Tomcat is installed and this is a simple one-time matter of creating a “build.properties” file with a single line on it. Something like this:

tomcat.home=C:/peter/opt/apache-tomcat-6.0.14

From the command prompt, change to the “myproject” root directory (which should contain build.xml and build.properties by now) and type the following command:

ant tomcat-start-debug

Once Tomcat starts you should be able to verify that the app was successfully deployed by pointing your browser to: http://localhost:8080/myproject/

Eclipse users can use the “mvn eclipse:eclipse” command to open the project in Eclipse and start working with the Ant targets.

The command to create a NetBeans free-form project from the Maven POM is as follows:

mvn antprops:nbfreeform

This will create the “nbproject” folder and the NetBeans project file as well as an extra Ant script for NetBeans integration targets. Now you can open “myproject” as a NetBeans free-form project.

The “tomcat-start-debug” target is conveniently mapped to the NetBeans “run” IDE action, so just clicking on the big green “play” button should deploy the web-app running on Tomcat. If you right click on the project root “myproject” node, a “Stop Tomcat” context menu item should helpfully appear as well.

You can try the clean, compile, and war Ant targets etc. – which should work. The Wicket archetype includes a sample JUnit test also in “TestHomePage.java”. After clicking on this file to select it within the NetBeans project explorer window – go to the Run –> Run File –> Test “TestHomePage.java” toolbar menu (or hit CTRL + F6) and you should be able to run only the selected test. But it will fail now with a “java.lang.NoClassDefFoundError: javax/servlet/ServletException”.

This problem actually gives us a good example for demonstrating how the “build-deps.properties” file can be updated when dependencies change or are added to the POM. Now add the missing Servlet API dependency to the “pom.xml” file by adding this snippet within the <dependencies> section:

<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.4</version>
    <scope>provided</scope>
</dependency>

Then re-run the command to generate build-deps.properties. Note that this command is designed so that if a “build.xml” file already exists, it will not be over-written.

mvn antprops:generate

Now run the test case again from NetBeans and it should pass. Unit testing a UI component is easy with Wicket!

One of the neat things about the Wicket archetype is that it embeds a Jetty web-app server using a small Java class called “Start.java” lying in the “test” source structure. The POM is already set with the right dependencies for Jetty and so you can simply right-click on this Java file within the NetBeans project explorer and choose “Run”. This will start a Jetty instance and deploy the WAR as well. You will be able to see the app running at http://localhost:8080/. Look ma, no Tomcat!

One more thing you can try is right-click on “Start.java” and do “Debug” instead of run. This will kick off a proper NetBeans debug session bringing up some sub-windows for viewing breakpoints, call stack etc. You can now try hot-deploying a single class without re-starting the app-server (Jetty). Try changing the text of the Label in HomePage.java to something else and saving the file. With the file selected in the NetBeans project window, look for the “Apply Code Changes” menu option – one place to find it is within the “Run” toolbar menu. A message should appear in the log saying that a class was reloaded. Refresh the browser and you should see the change in the text displayed.

Hot deploy should also work with Tomcat after initiating a debug-session (Attach Debugger) and then using the “Apply Code Changes” menu option.

The plugin is still experimental so if you find problems, do let me know in the comments. The commands for creating “native” NetBeans projects are “mvn antprops:nbjavase” and “mvn antprops:nbjavaee”. You need NetBeans with Java EE support to try the second option.

The plugin also should make it possible to package a project along with dependencies and hand-off to someone without Maven or internet access – but I haven’t completed this part yet.

Migrating to Apache Wicket – GIDS presentation slides

The ‘Migrating to Apache Wicket’ presentation I made at the Great Indian Developer Summit last week went off well. It was great to see the interest in Wicket, the room was full and there were quite a few questions even during the course of the session before I got to the end.

Here are the slides for your viewing pleasure. Apologies if some of the slides look cluttered because the animations I used to talk through some of the slides have been lost after converting to PDF.

Here below is one of the images I used to illustrate how a project that starts out using an action-oriented MVC framework with JSP can end up with a ‘Franken-stack’ over time. It starts innocently enough, you decide to use something extra say to enforce a common layout, then you realize you need some third party tag-libs, then security, and of course Ajax and before you know it…

What if a framework exists that can do all of this in Just Java and Just HTML? Find out more.

Hope you find the slides useful. I look forward to your feedback and suggestions on how the content can be improved. Please do comment!

How to record RTMP flash video streams using Red5

Update 2009-04-04: I’ve released an open source project that you can use to download RTMP video streams in a much simpler way. You can find it here: http://flazr.com

Red5 is an open source Flash server written in Java. It does not include a standalone client yet but I was able to write a Java program that uses Red5 to connect to an RTMP video stream and record / save it to a file. Code is provided below and also some tips on how to get the details required to download flash videos that you come across on the internet.

Here’s the code of “MyRtmpClient.java”, you are free to use and modify this in any way you want.

import java.io.File;

import org.apache.mina.common.ByteBuffer;
import org.red5.io.IStreamableFile;
import org.red5.io.ITag;
import org.red5.io.ITagWriter;
import org.red5.io.flv.impl.FLVService;
import org.red5.io.flv.impl.Tag;
import org.red5.io.utils.ObjectMap;
import org.red5.server.api.event.IEvent;
import org.red5.server.api.event.IEventDispatcher;
import org.red5.server.api.service.IPendingServiceCall;
import org.red5.server.api.service.IPendingServiceCallback;
import org.red5.server.net.rtmp.Channel;
import org.red5.server.net.rtmp.RTMPClient;
import org.red5.server.net.rtmp.RTMPConnection;
import org.red5.server.net.rtmp.codec.RTMP;
import org.red5.server.net.rtmp.event.AudioData;
import org.red5.server.net.rtmp.event.IRTMPEvent;
import org.red5.server.net.rtmp.event.Notify;
import org.red5.server.net.rtmp.event.VideoData;
import org.red5.server.net.rtmp.message.Header;
import org.red5.server.net.rtmp.status.StatusCodes;
import org.red5.server.stream.AbstractClientStream;
import org.red5.server.stream.IStreamData;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MyRtmpClient extends RTMPClient {

	private static final Logger logger = LoggerFactory.getLogger(MyRtmpClient.class);

	private String saveAsFileName = "test.flv";

	public static void main(String[] args) {

		String host = "localhost";
		String app = "oflaDemo";
		final String name = "IronMan.flv";
		int port = 1935;
		final int duration = 10000; // milliseconds, -2 means until end of stream

		final MyRtmpClient client = new MyRtmpClient();
		logger.debug("connecting, host: " + host + ", app: " + app + ", port: " + port);
		
		IPendingServiceCallback callback = new IPendingServiceCallback() {							
			public void resultReceived(IPendingServiceCall call) {
				logger.debug("service call result: " + call);
				if ("connect".equals(call.getServiceMethodName())) {					
					client.createStream(this);
				} else if ("createStream".equals(call.getServiceMethodName())) {													
					Integer streamId = (Integer) call.getResult();
					logger.debug("createStream result stream id: " + streamId);
					logger.debug("playing video by name: " + name);						
					client.play(streamId, name, 0, duration);
				}								
			}						
		};
		
		client.connect(host, port, app, callback);

	}

	private RTMPConnection conn;	
	private ITagWriter writer;

	private int videoTs;
	private int audioTs;

	@Override
	public void connectionOpened(RTMPConnection conn, RTMP state) {
		logger.debug("connection opened");
		super.connectionOpened(conn, state);
		this.conn = conn;
		init();
	}

	@Override
	public void connectionClosed(RTMPConnection conn, RTMP state) {
		logger.debug("connection closed");
		super.connectionClosed(conn, state);
		if (writer != null) {
			writer.close();
			writer = null;
		}
		System.exit(0);
	}

	@Override
	public void createStream(IPendingServiceCallback callback) {
		logger.debug("create stream");
		IPendingServiceCallback wrapper = new CreateStreamCallBack(callback);
		invoke("createStream", null, wrapper);
	}

	@Override
	protected void onInvoke(RTMPConnection conn, Channel channel, Header header, Notify notify, RTMP rtmp) {
		super.onInvoke(conn, channel, header, notify, rtmp);		
		ObjectMap<String, String> map = (ObjectMap) notify.getCall().getArguments()[0];
		String code = map.get("code");
		if (StatusCodes.NS_PLAY_STOP.equals(code)) {
			logger.debug("onInvoke, code == NetStream.Play.Stop, disconnecting");
			disconnect();
		}
	}	

	private void init() {
		File file = new File(saveAsFileName);
		FLVService flvService = new FLVService();
		flvService.setGenerateMetadata(true);
		try {
			IStreamableFile flv = flvService.getStreamableFile(file);
			writer = flv.getWriter();
		} catch (Exception e) {
			throw new RuntimeException(e);
		}
	}

	private class CreateStreamCallBack implements IPendingServiceCallback {

		private IPendingServiceCallback wrapped;

		public CreateStreamCallBack(IPendingServiceCallback wrapped) {
			this.wrapped = wrapped;
		}

		public void resultReceived(IPendingServiceCall call) {
			Integer streamIdInt = (Integer) call.getResult();
			if (conn != null && streamIdInt != null) {
				MyNetStream stream = new MyNetStream();
				stream.setConnection(conn);
				stream.setStreamId(streamIdInt.intValue());
				conn.addClientStream(stream);
			}
			wrapped.resultReceived(call);
		}

	}

	private class MyNetStream extends AbstractClientStream implements IEventDispatcher {

		public void close() { }

		public void start() { }

		public void stop() { }

		public void dispatchEvent(IEvent event) {
			if (!(event instanceof IRTMPEvent)) {
				logger.debug("skipping non rtmp event: " + event);
				return;
			}
			IRTMPEvent rtmpEvent = (IRTMPEvent) event;
			if (logger.isDebugEnabled()) {
				logger.debug("rtmp event: " + rtmpEvent.getHeader() + ", "
						+ rtmpEvent.getClass().getSimpleName());
			}
			if (!(rtmpEvent instanceof IStreamData)) {
				logger.debug("skipping non stream data");
				return;
			}
			if (rtmpEvent.getHeader().getSize() == 0) {
				logger.debug("skipping event where size == 0");
				return;
			}
			ITag tag = new Tag();
			tag.setDataType(rtmpEvent.getDataType());
			if (rtmpEvent instanceof VideoData) {				
				videoTs += rtmpEvent.getTimestamp();
				tag.setTimestamp(videoTs);
			} else if (rtmpEvent instanceof AudioData) {
				audioTs += rtmpEvent.getTimestamp();
				tag.setTimestamp(audioTs);
			}
			ByteBuffer data = ((IStreamData) rtmpEvent).getData().asReadOnlyBuffer();
			tag.setBodySize(data.limit());
			tag.setBody(data);
			try {
				writer.writeTag(tag);
			} catch (Exception e) {
				throw new RuntimeException(e);
			}
		}
	}

}

To compile and run the code you need the following libraries:

commons-collections-3.2.jar
jcl104-over-slf4j-1.4.3.jar
logback-classic-0.9.8.jar
logback-core-0.9.8.jar
mina-core-1.1.6.jar
red5.jar
slf4j-api-1.4.3.jar
spring-beans-2.0.8.jar
spring-context-2.0.8.jar
spring-core-2.0.8.jar

We need the latest Red5 JAR built from version control (not the latest official release) and I have uploaded these files here: red5-rtmp-client-libszip.pdf – just rename the file to end with “.zip” after downloading.

The code shown is hard-coded to connect to “localhost”, app name “oflaDemo” and stream name “IronMan.flv” which can be tested on the official Red5 server distribution which you can download and run. When it comes to downloading flash streams from the internet, WireShark can be used to sniff out the values you need.

Let’s take this site for example: http://videolectures.net/ Start a WireShark capture session before clicking on a video on the site to play it. Let WireShark grab all the information exchanged between your PC and the remote flash server and you can stop the capture once the video begins to play, we are only interested in what goes on during the connection handshake. I will use this video as an example: http://videolectures.net/ff06_chomsky_szmp/ [update Jun-2008: looks like they changed this particular video to Windows media instead of Flash, so try other videos or other sites]

In WireShark you can filter for protocol “rtmpt” and the first few entries would be handshake or “invoke” operations. Examining the “Handshake part 3″ we can easily get the value of the required “app” property. Below we can see it is “video/2006/other/ff06/chomsky_noam”:

For the host name, the IP address should do fine for most sites, but we can easily figure out the host name of the stream server from what appears after “rtmp://”. Note that WireShark allows you to search the text contents of captured packets. Here below we can see that the host name is “velblod.videolectures.net”:

And finally when the “play” command is issued – we need the value of the stream name. Below we see it is “chomsky_noam_01″:

So with the right values of hostname, app and stream name set – you can run the program and download the stream to your local drive for offline viewing. To download the whole stream – just change the duration to ‘-2′ as hinted in the source code comment. There are many free Flash players available you can use to play downloaded content such as FLV Player.

Do let me know if this works for you and if you find any additional parameters that need to be passed for other sites.

Never install NetBeans again! (just use the ZIP)

Okay that title was just to grab your attention :) What I mean is that I’m going to show you how to use the NetBeans ZIP distribution on Windows so that you never need to use the installer executable again. If you are like me and would like total control over the folders that NetBeans uses, or if you hate installing things onto Windows because of all that stuff that gets added to the programs menu and your user profile folders and maybe your registry – you will probably like this approach. Now that NetBeans 6.1 release candidate
is out, this is a good way for you to try it out in an un-intrusive manner especially if you have an older version of NetBeans already installed.

Although most people use the platform specific installers, NetBeans can also be downloaded as a platform-independent ZIP file. You can get the various flavors here for 6.1 RC: http://dlc.sun.com.edgesuite.net/netbeans/6.1/rc/zip/

I am going to use the “javase” version which is just 50MB. I can’t help mentioning that two years back, one of the main reasons I switched to NetBeans was because I felt that Eclipse was getting too bloated for comfort. Never regretted it, and especially now that Apache Wicket is my web-framework of choice – the NetBeans “javase” version is just perfect because Wicket only needs HTML and Java editing support.

After downloading the file, in this case called “netbeans-6.1rc1-200804100130-javase.zip” extract it to any folder of your choice. Create a batch file (you can call it “netbeans.bat”) and place it in the top level “netbeans” folder itself with the following contents:

set NETBEANS_HOME=%~dp0
set NETBEANS_HOME=%NETBEANS_HOME:~0,-1%

set JAVA_HOME=C:\peter\app\jdk1.6.0_02
set NETBEANS_USER_DIR=C:\peter\workspaces\netbeans

start /b %NETBEANS_HOME%\bin\netbeans.exe --jdkhome %JAVA_HOME% --userdir %NETBEANS_USER_DIR%

The first two lines are DOS hacks to obtain the path of the current directory (without a trailing slash) where you un-zipped NetBeans. You should point the JAVA_HOME to the JDK you wish to use. And finally, you can customize the folder where NetBeans saves your settings (which by default goes somewhere into your “Documents and Settings” folder). The “start /b” part is a way to make the console window “disappear” after NetBeans launches otherwise you may terminate it later and kill NetBeans by mistake.

That’s it! Now just double-click on the batch file and NetBeans will start and even create the “userdir” if it does not exist. I actually use NetBeans like this on a daily basis. It should be easy to evaluate NetBeans this way – you don’t have to go through any un-install step (in the unlikely event that you don’t like NetBeans :)

This is a nice way to control which JDK NetBeans uses, which is useful in case you have multiple JDKs installed. I use this approach for conducting training workshops where I provide a ZIP that includes NetBeans, a custom bundled JDK and a sample project – the batch file uses relative paths, just unzip and you are good to go.

If you have a NetBeans tip to share – enter the NetBeans blogging contest here: http://www.netbeans.org/competition/blog-contest.html?cid=923686

Fibonacci spiral fun with Java3D

Recently I became fascinated with the Fibonacci series and the Golden Ratio, especially with how they turn up in nature. I had been meaning to explore the Java3D API for a while, so I thought it would be a good idea to try and generate some Fibonacci spirals and get to know Java3D that way. Here is one example of what I could come up with.

fibo_java3d_1

I was able to tweak the code to highlight interesting patterns. I actually thought for a moment that I had “discovered” a few things like for instance how if you number the “dots” by the order in which they are introduced into the simulation, the spirals that visually strike you are formed by joining the dots where the number difference is a fibonacci number.

Of course all this is common knowledge in the mathematics community. But I think I was able to create much prettier visualizations :) Here is a snapshot showing how the two most obvious spirals are where the dot-number difference is equal to 13 and 21 (successive Fibonacci numbers). Dot number 1 is on the far right.

fibo_java3d_2

And here is this interesting effect when the dots are numbered in reverse order, the dots corresponding to the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144) line up along the x-axis.

fibo_java3d_3

Well, this is animation after all – so here is my first ever YouTube video :) If you are interested, here is the source code (not much documentation I’m afraid).

I found the fantastic sunflower photo on Flickr. The photo description has a good explanation of why the fibonacci spirals happen and the maths behind it. If you want to explore more, have a look at this Science News Online article.

Update 2008-12-13: Found this link to some nice mathematical analysis of number spirals (not Fibonacci): http://www.numberspiral.com/

A Wicket user tries JSF

A year or so back I attempted to learn JSF and even bought one of the recommended books. But after reading through the first hundred or so pages, I came to the conclusion that it was too complicated and un-intuitive. The code samples looked ugly to me in a way that was difficult to explain. The fact that there was a different expression language (with a ‘#’ instead of ‘$’) to get used to when I was just getting comfortable with JSP EL did not help either.

So I gave up trying to learn JSF and decided to concentrate on Spring MVC at that time.

Fast forward to today and the main trigger for my current experiments with JSF is because after my recent blog post on migrating from Spring MVC / WebFlow to Wicket, I am increasingly being asked whether I ever tried JSF.

So I have attempted to write a very simple application (based on MyFaces) and I will try to compare it side by side with the exact same functionality implemented in Wicket. I just started with JSF a couple of days back so feel free to comment and let me know if I missed something fundamental and I will update things either in this post or the next.

Functionality
This is supposed to be a “discussion forum” application and there is only one domain object called “Forum” with two String fields “name” and “description”. Maybe in the future I will include “Topic” and “Message” objects but for now I’m trying to keep things as simple as possible.

screens.png

There are only two screens. The home page is a list of Forum(s) in the system. There is a link to create a new Forum. This brings up the other screen which is a form with two text fields and a submit button. After a successful submission, you are taken back to the home page and the newly created Forum appears in the list.

Environment
NetBeans 5.5 is being used. I deploy to a Jetty instance that I set up manually because I want to also compare the Jetty “footprint” required for Wicket vs MyFaces. With some digging I’ve tried to arrive at the bare minimum “quickstart” config required, for example the web.xml file cannot be possibly smaller. I’m using Maven2 directory and naming conventions. Defaults are used as far as possible. In the side by side comparison images below JSF is on the left and Wicket is on the right.

So here goes:

Project Structure
project-structure.png

  • Wicket does not require any extra XML config like you have faces-config.xml for JSF
  • Wicket does not require JSPs
  • By default, Wicket expects the markup (*.html files) to be side by side with the *.java files corresponding to components – e.g. Page(s). Some people find this a bit odd but this keeps things simple and there are some very good reasons for this. You can change this convention if you really want to.
  • The MyFaces application requires an “index.jsp” welcome-file to redirect to the virtual home page URL

Jetty Footprint
jetty-footprint.png
Since Wicket does not require JSP support – the Jetty footprint is much smaller – the contents of the Jetty “lib” folder come down from 6.16 MB to 692 KB

Library Dependencies
dependencies.png
Even without using Tomahawk etc., MyFaces requires far more dependencies than Wicket i.e. 10 vs 4 JAR files

faces-config.xml
faces-config-xml.png
Wicket does not require any kind of XML configuration at all and the “faces-config.xml” file does not have any equivalent. Navigation logic is performed in pure Java code which is much more flexible and type-safe and you don’t need to synchronize multiple files.

web.xml
web-xml.png
A servlet filter handles all the framework stuff in Wicket. This has some advantages like you can map the root of your context path “/” to the home page. Anything not meant for the Wicket filter like images, css reources etc. will be ignored by the filter.

You need to tell Wicket where your “Application” class is located. This Application class is used primarily to point to the designated home page.

The Wicket Application Class
wicket-application-class.png
In a real life application, this is where you would manage references to the service-layer, configure security and various other things. Here we are just configuring the home page.

The “List Forums” Screen
forum-list-screen2.png

  • The wicket markup is pure HTML but the MyFaces JSP is a mixture of JSP, HTML, JSF tags and JSF EL
  • In the case of Wicket you can see that the complexity clearly moves out of the markup into the Java code. In the JSF version, the only thing the backing class is doing is setting up the model / list
  • You don’t need a specialized IDE to edit the Wicket version – HTML editing and Java editing support is sufficient. In the case of JSF you have to rely on IDE support when the UI gets complex
  • If you make a spelling mistake in the JSF EL expressions – you won’t know until run time – but the Java code that renders dynamic data in the Wicket version does not have that problem, you benefit from your IDE syntax coloring, refactoring suppport etc.
  • The JSF version is using a “dataTable” control. For Wicket we are using a ListView repeater control and creating a table manually. Even then, this is very simple code and you have much more control over the markup. The JSF dataTable documentation is not for the faint of heart and runs into a few pages. That is a lot of complexity you don’t need most of the time. BTW Wicket does provide a range of grid controls in the Wicket extensions project
  • Instead of navigation rules defined in faces-config.xml and the need to synchronize this with the action /event names you use in your JSP or backing class – in Wicket you can easily navigate from page to page by using the right “Link” components or setResponsePage()
  • Instead of using “id” which has a lot of significance in terms of (X)HTML – Wicket uses a namespace so “wicket:id” is the attribute you use to link your Java code and markup. Relying on the “id” attribute can lead to all kinds of unpleasant name collisions in JSF.

The “Create New Forum” Form
forum-form-screen3.png
In addition to the points in the previous section:

  • In JSF you declare and configure components in the markup and you have to use the right tags (e.g. h:form, h:inputText, h:commandButton) and the right attributes (e.g. to mark an input field as “required”). JSF is supposed to be a component oriented framework but I wouldn’t call them Java components and definitely not Object Oriented.
  • In Wicket you use normal HTML tags in the markup and in the Java code, you instantiate and attach the core components (e.g. TextField) or extend them as required (e.g. Form). All the behavior, and things like setting a field as “required” happen in pure Java code.
  • In this case for Wicket the value of the “wicket:id” attribute on the input text fields also serves as OGNL notation for HTML form –> POJO binding – which reduces the amount of configuration.
  • Rather than rely on a h:panelGrid to layout the form fields it is easier and more intuitive to use plain old HTML tables for this purpose.
  • The screenshot below demonstrates how the “ForumFormPage.html” file looks like when opened in a browser. Being valid HTML, Wicket markup files can be previewed and edited as such. Wicket provides for true separation of concerns and HTML screen design and layout can *really* be done by a team separate from the Java development team

browser-preview.png

Generated HTML
The HTML generated by MyFaces is unbelievably verbose and contains all kind of strange javascript which I never, ever would have expected. If there is one reason to stay away from MyFaces, this is it.

You can click on the links below to see the output. If your browser renders the HTML instead of displaying the raw text, just do “view source”.

For Wicket, the generated HTML is clean like you would expect. And note that the Wicket specific attributes like “wicket:id” will be stripped in PRODUCTION mode.


Update [2007-06-10]: Simplified the wicket code for the “Create New Forum” form after it dawned on me that the existing POJO is more than sufficient for form binding. The older version can be found here: forum-form-screen2.png

The Java code for the Wicket form page can be made even more tighter using an anonymous inner class for the form. This is a nice example of how Wicket gets you to exercise the Java OO parts of your brain, and it is worth comparing this with the “ForumFormPage.java” above.

Update: I uploaded the Java and HTML source for the 2 Wicket pages. Since WordPress has restrictions on uploads (no *.zip allowed for e.g.), you have to rename the files correctly if you download them – Forum List Page: HTML / Java | Forum Form Page: HTML / Java

How to do String operations in Ant

I was trying to create an Ant based wizard to generate a Wicket quickstart project. So like the trend nowadays the script should prompt you for the project name at the start then *boom* – you get your Maven2 directory structure, pom.xml, web.xml, a couple of placeholder web-pages – just type mvn jetty:run and you are up and running.

I wanted the user to only enter the project-name: a lower-case string without spaces or special characters. Then I wanted the Wicket Application class to be generated as per tradition i.e. if your project short name is “test” the file should be “TestApplication.java”

But then it hit me – there is no way to do things like uppercase / lowercase string conversion in Ant! Yikes. I could make-do with a second prompt for the user to get the application class name, but hey, that’s so not-DRY, and I like an Ant challenge :)

One option was to use Beanshell – but I’m obsessed with getting things to work with a “clean” Ant install, without messing around with the classpath or ant-home. After being spoilt by Maven, I don’t like having orphan jar files lying around in the project directory structure either.

So I ended up doing this. Here is a small Java class that processes a command line argument and writes the result to a file:

import java.io.FileOutputStream;
import java.io.PrintWriter;

public class TitleCaser {

    public static void main(String[] args) throws Exception {
        FileOutputStream os = new FileOutputStream("target/temp.txt");
        PrintWriter out = new PrintWriter(os);
        String s = args[0];
        out.write(Character.toUpperCase(s.charAt(0)) + s.substring(1));
        out.close();
        os.close();
    }

}

This java file resides in an “etc” folder. And the start of the Ant script looks like this:

<javac srcdir="etc" includes="TitleCaser.java"/>        
<input message="Enter Project Name:" addproperty="project.name"/>
<java classpath="etc" classname="TitleCaser">
     <arg value="${project.name}"/>
</java>
<loadfile property="project.name.titleCase" srcFile="target/temp.txt"/> 

I know, I know, Ant is XML, it is a declarative language, I must be crazy etc. But I can think of all kinds of interesting things to do with this approach. There are various ways to load stuff into Ant from files, “loadfile” is just one of them. Think of the possibilities – you can generate Java code from Ant, “javac” it on the fly and work around all the Ant limitations like not having good control-flow support etc. Maybe you can even generate some XML and use that in a downstream target and tell your boss that your Ant build is SOA compliant…

Follow

Get every new post delivered to your Inbox.

Join 29 other followers