Comments on: Watij and running JUnit test methods in the order that you want http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/ Peter Thomas Thu, 05 Nov 2009 03:10:42 +0000 http://wordpress.com/ hourly 1 By: 6as0fb http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-13632 6as0fb Sun, 17 May 2009 06:26:00 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-13632 asdsadadsa asdsadadsa

]]> By: Click http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-12678 Click Tue, 15 Apr 2008 22:30:19 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-12678 Selenium RC IS much better. Good move. Selenium RC IS much better. Good move.

]]> By: naisioxerloro http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-11909 naisioxerloro Wed, 28 Nov 2007 21:15:16 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-11909 Hi. Good design, who make it? Hi.
Good design, who make it?

]]> By: Peter Thomas http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-8993 Peter Thomas Tue, 24 Jul 2007 14:31:11 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-8993 Yep, I recently switched to Selenium RC and it is much better than Watij. Maybe a detailed blog post on that later. Yep, I recently switched to Selenium RC and it is much better than Watij. Maybe a detailed blog post on that later.

]]> By: ivan http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-8991 ivan Tue, 24 Jul 2007 10:13:39 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-8991 I would actually prefer to use Selenium RC for the https protocol which can be used via junit as I find its extremely easy to implement compared to watij -- anyways thats my side of the coin :) I would actually prefer to use Selenium RC for the https protocol which can be used via junit as I find its extremely easy to implement compared to watij — anyways thats my side of the coin :)

]]> By: SS http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-405 SS Tue, 29 Aug 2006 19:03:47 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-405 My bad -- pls. ignore all the private method stuff. I shouldve read your post more carefully BEFORE replying (duh!) :| My bad — pls. ignore all the private method stuff. I shouldve read your post more carefully BEFORE replying (duh!) :|

]]> By: SS http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-404 SS Tue, 29 Aug 2006 18:33:17 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-404 We have run into the problem Jim indicated above. Development was on Windows boxes and the CI environment was on AIX. The AIX JDK returned the methods in exactly the reverse order to that by the Sun JDK on Windows! Bottom-line - believe the Javadoc -- dont depend on the reflected order :) Similarly, if the Suite.addTest contract doesnt indicate that they will be run in order -- even though the current implementation might be running it in-order, the implementors are free to change that if they want to. Also, if all you always want to do is run all the tests every time - have 1 public test method and call your individual private test methods in the order you want. Yes - the order goes into the code and you have to recompile to change - but if you are always running the full suite - what difference! (BTW, as unit-test-purists will point out - you are running integration tests here not unit-tests per se and hence your test-dependencies. In short (ok its too late for that :) ) -- dont blame the framework since it was not meant for those ends! ) We have run into the problem Jim indicated above. Development was on Windows boxes and the CI environment was on AIX. The AIX JDK returned the methods in exactly the reverse order to that by the Sun JDK on Windows! Bottom-line – believe the Javadoc — dont depend on the reflected order :)

Similarly, if the Suite.addTest contract doesnt indicate that they will be run in order — even though the current implementation might be running it in-order, the implementors are free to change that if they want to.

Also, if all you always want to do is run all the tests every time – have 1 public test method and call your individual private test methods in the order you want.
Yes – the order goes into the code and you have to recompile to change – but if you are always running the full suite – what difference!

(BTW, as unit-test-purists will point out – you are running integration tests here not unit-tests per se and hence your test-dependencies. In short (ok its too late for that :) ) — dont blame the framework since it was not meant for those ends! )

]]> By: Jake Dempsey http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-401 Jake Dempsey Fri, 25 Aug 2006 14:50:44 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-401 Peter, Glad to hear you have decided to use Watij. Let us know if you have any quesions. If you haven't already, make sure to sign up on the Google Group at http://groups.google.com/group/watij. Let us know if you have any questions. Peter,

Glad to hear you have decided to use Watij. Let us know if you have any quesions. If you haven’t already, make sure to sign up on the Google Group at http://groups.google.com/group/watij. Let us know if you have any questions.

]]> By: Peter Thomas http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-398 Peter Thomas Thu, 24 Aug 2006 17:51:06 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-398 It is working fine on a JUnit test case with quite a few methods on it. :\ $%&*# Java Specification :). Maybe this is why even JUnit 4 and TestNG don't provide a simple way to do what I want. Really appreciate the feedback Jim :) <em>[update: For now, as I said it is working fine for me (JDK 1.5.0_06 - Windows XP SP2 -JUnit 3.8.1) and I will update this post in case I see any need to change this approach.]</em> So maybe this approach needs more work to be foolproof. For ordering of the methods I'll probably try something like what the Ruby unit test framework does, which is to sort the test methods alphanumerically. So maybe I'll end up naming the methods test10_DoSomething, test20_DoSomethingElse etc. I really hope you are wrong on the TestSuite.addTest() concept, and I've seen in <a href="http://www.awprofessional.com/articles/article.asp?p=101374&seqNum=5&rl=1" rel="nofollow">quite</a> a <a href="http://www.javaworld.com/javaworld/jw-12-2000/jw-1221-junit.html" rel="nofollow">few</a> places that the order is guaranteed. Oh well I could always extend TestSuite... It is working fine on a JUnit test case with quite a few methods on it. :\ $%&*# Java Specification :).

Maybe this is why even JUnit 4 and TestNG don’t provide a simple way to do what I want.

Really appreciate the feedback Jim :)

[update: For now, as I said it is working fine for me (JDK 1.5.0_06 - Windows XP SP2 -JUnit 3.8.1) and I will update this post in case I see any need to change this approach.]

So maybe this approach needs more work to be foolproof. For ordering of the methods I’ll probably try something like what the Ruby unit test framework does, which is to sort the test methods alphanumerically. So maybe I’ll end up naming the methods test10_DoSomething, test20_DoSomethingElse etc.

I really hope you are wrong on the TestSuite.addTest() concept, and I’ve seen in quite a few places that the order is guaranteed.

Oh well I could always extend TestSuite…

]]> By: Jim http://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-397 Jim Thu, 24 Aug 2006 15:09:38 +0000 https://ptrthomas.wordpress.com/2006/08/23/watij-and-running-junit-test-methods-in-the-order-that-you-want/#comment-397 (Hopefully) constructive feedback: I don't think that you solved your problem. It may work, but it's not (from what I read) guaranteed by the docs. From the documentation for getMethods(), it says "The elements in the array returned are not sorted and are not in any particular order." http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Class.html#getMethods() And in the TestSuite class, I can't find it saying that the order in which tests are added is the order that it will be performed in. http://junit.sourceforge.net/javadoc/junit/framework/TestSuite.html#addTest(junit.framework.Test) I'm not knocking that what you wrote works for your needs, but just to let you know that if the implementation of getMethods or JUnit is different from what you're working with, your code *could* break (but it's not likely). In an older version of a java testing framework (strutstestcase? junit? cactus? I can't rememeber) it would run the tests from the bottom of the java file to the top. I upgraded the lib and they ran top down. :-( (Hopefully) constructive feedback:

I don’t think that you solved your problem. It may work, but it’s not (from what I read) guaranteed by the docs.

From the documentation for getMethods(), it says “The elements in the array returned are not sorted and are not in any particular order.”
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Class.html#getMethods()

And in the TestSuite class, I can’t find it saying that the order in which tests are added is the order that it will be performed in.
http://junit.sourceforge.net/javadoc/junit/framework/TestSuite.html#addTest(junit.framework.Test)

I’m not knocking that what you wrote works for your needs, but just to let you know that if the implementation of getMethods or JUnit is different from what you’re working with, your code *could* break (but it’s not likely).

In an older version of a java testing framework (strutstestcase? junit? cactus? I can’t rememeber) it would run the tests from the bottom of the java file to the top. I upgraded the lib and they ran top down. :-(

]]>