Watij and running JUnit test methods in the order that you want

I did consider using Selenium, but I’m going with Watij for the moment – if you can live with the “IE only” limitation, it is faster and much less hassle to get up and running. The new 3.0 version uses a much better engine under the covers and appears to be much faster than before. I sure hope that the Watij team deliver on their promise to have FireFox support sometime in the
future.

I haven’t yet bothered to move to JUnit 4 or TestNG and I had got this JUnit (3.8.1) test script working that would open an IE instance and run through a series of application screens, executing a good bunch of functionality. The entire script was packed into a single JUnit method because of the well known JUnit “feature” which is that the order in which test methods are executed at runtime is not guaranteed.

I wanted to split my JUnit test script into separate self documenting methods which I felt would be more readable. Also I wanted to make the JUnit report give a good feel of the functionality executed by listing all the test method names executed along with the time taken for each.

This is what I came up with that tries to solve the problem of ensuring the order of test method execution using reflection. A “global” instance of the Internet Explorer object is maintained using a ThreadLocal which is preserved across JUnit test method executions.

Here is the abstract base class:

import java.lang.reflect.*;
import junit.framework.*;
import watij.runtime.ie.IE;

public abstract class WatijTestCase extends TestCase {
    
    public WatijTestCase(String name) {
        super(name);
    }
            
    private static ThreadLocalIE threadLocalIE;
    protected static Class clazz;
    protected IE ie;
    
    public static Test suite() throws Exception {        
        threadLocalIE = new ThreadLocalIE();
        Constructor constructor = clazz.getDeclaredConstructors()[0];
        Method[] methods = clazz.getMethods();
        TestSuite s = new TestSuite();
        for(Method m : methods) {
            if (m.getName().startsWith("test")) {
                Test test = (Test) constructor.newInstance(new Object[] { m.getName() });
                s.addTest(test);
            }
        }
        return s;
    }
    
    private static class ThreadLocalIE extends ThreadLocal {
        @Override
        public IE initialValue() {
            return new IE();
        }
    }
    
    @Override
    public final void setUp() {
        ie = (IE) threadLocalIE.get();
    }    
    
}

And here is the actual test (with this curious hack of a static block :)

public class AllTest extends WatijTestCase {
    
    static {
        clazz = AllTest.class;
    }
    
    public AllTest(String name) {
        super(name);
    }        
    
    public void testGetLoginPage() throws Exception {
        ie.start("http://localhost:8080/jtrac");
        assertTrue(ie.containsText("JTrac"));
    }
    
    public void testSuccessfulLogin() throws Exception {        
        ie.textField(name, "j_username").set("admin");
        ie.textField(name, "j_password").set("admin");
        ie.button("Submit").click();
        assertTrue(ie.containsText("DASHBOARD"));
    }     
    
    public void testCreateNewSpaceAndAllocateAdmin() throws Exception {
        
        ie.link(text, "OPTIONS").click();
        assertTrue(ie.containsText("Options Menu"));        
        
        ie.link(text, "Spaces").click();
        assertTrue(ie.containsText("Spaces"));
        ....
        ....
    }
}

Would be great to hear suggestions on how to improve this.

[Related earlier post on Watir and discovering Watij.]

12 Responses to Watij and running JUnit test methods in the order that you want

  1. x says:

    If you need to run your test methods in a certain order, TestNG is definitely the way to go since it supports this natively (and it even lets you specify that methods depends on groups of methods, which is very powerful).

  2. Peter Thomas says:

    Actually I had browsed through the features of both TestNG and JUnit 4 and I was not satified with how you can specify test dependencies in either. What I understood is that if you want to specify the order in which test methods execute, you have to annotate *every* method. Did I miss something?

    I prefer the approach outlined in this blog post – it is far more easier and intuitive to expect the tests to run in the order they appear in the java code.

  3. Jim says:

    (Hopefully) constructive feedback:

    I don’t think that you solved your problem. It may work, but it’s not (from what I read) guaranteed by the docs.

    From the documentation for getMethods(), it says “The elements in the array returned are not sorted and are not in any particular order.”
    http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Class.html#getMethods()

    And in the TestSuite class, I can’t find it saying that the order in which tests are added is the order that it will be performed in.
    http://junit.sourceforge.net/javadoc/junit/framework/TestSuite.html#addTest(junit.framework.Test)

    I’m not knocking that what you wrote works for your needs, but just to let you know that if the implementation of getMethods or JUnit is different from what you’re working with, your code *could* break (but it’s not likely).

    In an older version of a java testing framework (strutstestcase? junit? cactus? I can’t rememeber) it would run the tests from the bottom of the java file to the top. I upgraded the lib and they ran top down. :-(

  4. Peter Thomas says:

    It is working fine on a JUnit test case with quite a few methods on it. :\ $%&*# Java Specification :).

    Maybe this is why even JUnit 4 and TestNG don’t provide a simple way to do what I want.

    Really appreciate the feedback Jim :)

    [update: For now, as I said it is working fine for me (JDK 1.5.0_06 – Windows XP SP2 -JUnit 3.8.1) and I will update this post in case I see any need to change this approach.]

    So maybe this approach needs more work to be foolproof. For ordering of the methods I’ll probably try something like what the Ruby unit test framework does, which is to sort the test methods alphanumerically. So maybe I’ll end up naming the methods test10_DoSomething, test20_DoSomethingElse etc.

    I really hope you are wrong on the TestSuite.addTest() concept, and I’ve seen in quite a few places that the order is guaranteed.

    Oh well I could always extend TestSuite…

  5. Jake Dempsey says:

    Peter,

    Glad to hear you have decided to use Watij. Let us know if you have any quesions. If you haven’t already, make sure to sign up on the Google Group at http://groups.google.com/group/watij. Let us know if you have any questions.

  6. SS says:

    We have run into the problem Jim indicated above. Development was on Windows boxes and the CI environment was on AIX. The AIX JDK returned the methods in exactly the reverse order to that by the Sun JDK on Windows! Bottom-line – believe the Javadoc — dont depend on the reflected order :)

    Similarly, if the Suite.addTest contract doesnt indicate that they will be run in order — even though the current implementation might be running it in-order, the implementors are free to change that if they want to.

    Also, if all you always want to do is run all the tests every time – have 1 public test method and call your individual private test methods in the order you want.
    Yes – the order goes into the code and you have to recompile to change – but if you are always running the full suite – what difference!

    (BTW, as unit-test-purists will point out – you are running integration tests here not unit-tests per se and hence your test-dependencies. In short (ok its too late for that :) ) — dont blame the framework since it was not meant for those ends! )

  7. SS says:

    My bad — pls. ignore all the private method stuff. I shouldve read your post more carefully BEFORE replying (duh!) :|

  8. ivan says:

    I would actually prefer to use Selenium RC for the https protocol which can be used via junit as I find its extremely easy to implement compared to watij — anyways thats my side of the coin :)

  9. Peter Thomas says:

    Yep, I recently switched to Selenium RC and it is much better than Watij. Maybe a detailed blog post on that later.

  10. naisioxerloro says:

    Hi.
    Good design, who make it?

  11. Click says:

    Selenium RC IS much better. Good move.

  12. 6as0fb says:

    asdsadadsa

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: