Both
Selenium and
WatiN have been around for a while, and both are fairly good tools when it comes to UI automated testing. Both tools now drive both IE7 and FF2, and are free to download. I have been using Selenium RC for a while now, and it seems to be fairly robust, although sometimes rather frustrating and somewhat slow. Both have a recorder mechanism, and both emit C# code for me. However the WatiN recorder did blow sky high with an unhandled exception when a pop-up window came up, but
the recorder code was written by someone else so I won't hold it against WatiN.
I tried WatiN for the first time today, and decided to go with the V.2 CTP instead of the .NET 1.1 release. It seems that some of the API's have been refactored into more sensible structures (factory patterns, etc.), but somehow the things that would have made the most sense to me in the API were structured much differently that I would have expected. There were some problems with my IE (it kept crashing unless I ran the debug version of W2). However, I blame this on IE, not on W2. FF has a plug-in that needs to be manually installed. It comes with the CTP, but doesn't get installed. Hopefully this will be integrated by release.
Most of the basic features are similar in function but not implementation. I can see writing an interface class and a couple of wrapper classes that implement the interface, one each for WatiN and Selenium, so we can have the best of both worlds.
WatiN Pro's
- No proxy server required. It just runs. Runs both FF and IE, switchable with the parameter to the factory method that creates the instance.
WatiN Con's
- Documentation is less than sparse.
- I could not find a way to do a "drag and drop" which is easily accomplished with Selenium.
- Does not handle pop-up windows. This was the show-stopper for me.
Selenium Pro's
- polished code
- well-documented functionality
- lots of examples
- handles lots of weird types of conditions and situations (most of which we will all need at some point.)
Selenium Con's
- have to run a java proxy in a separate process and set it up and tear it down
- slow in places, due to proxy and not-so-well-written sections of code
- locators aren't always easy to figure out. Tools like Firebug and XPather help identify locator strings.
Since it does not handle pop-up windows (or I just couldn't figure it out in about 4h of research), this was the end of the line for it. Selenium does handle them, and has been able to accomplish 100% of the weird scenario tests I have needed so far. Loads of tweaking and frustrating trial and error notwithstanding.
So, until we have a bit closer to release for WatiN, Selenium still wins (for now). However, I definitely plan to write the interface and shim both out so I can use either or both later on. Stay tuned for that posting, that will be the money shot...
In closing, let me just reiterate that every developer should be thinking about UI automated testing for web code. Each user-interactive element on the page NEEDS AN ID. That's every link, every button, every text box, checkbox, radio button, and other user input elements. The ID's need to be simple, and either fixed, or predictable. With these things in place in the production code, it will be testable and easier to validate. Better quality code will make it out to the hands of the users, and all will be right with the world. Amen.
<steps down from soapbox./>