In my last post I had a look at a little-known Integration testing framework called Fitnesse.  In this post I’m going to look at one of the more well-known ones – Selenium.

Selenium is a pure web-orientated integration testing platform.  The framework consist of 2 main components – an IDE and a server called Selenium Remote Control (RC).

Selenium IDE is a Firefox add-on that records clicks, typing, and other actions to make a test, which you can play back in the browser.  Tests can be saved in a number of different languages, including C#.  If you choose the C# option the tests will be saved as NUnit tests.  The tests are then executed in multiple browsers using Selenium RC.  An extension to Selenium RC called Selenium Grid allows you to run multiple tests in parallel.  There is a very nice 2-minute introduction on the Selenium website.

One of the first things you’ll notice about Selenium is the lack of proper documentation.  As with many open-source projects the documentation seems to go missing along the way.

Example

To illustrate the magic of Selenium, I’ve created a very simple website that calculates the user’s age based on their year of birth and the current year.

Amazing – don’t you sometimes wish there was such a website?

I’m also going to create a test using the Selenium IDE to check these values.  Recording a test is very simple – you fire up the IDE, click record and do whatever you want to test.  The only difference is that I am going to verify the result being returned from the server.

Nothing to it.  You can also see the code being generated as you record the test – pretty cool.  I now simply choose to export the tests as C#.

[TestFixture]
public class NewTest
{
    private ISelenium selenium;
    private StringBuilder verificationErrors;

    [SetUp]
    public void SetupTest()
    {
        selenium = new DefaultSelenium("localhost", 4444, "*chrome", "http://localhost:57800/");
        selenium.Start();
        verificationErrors = new StringBuilder();
    }

    [TearDown]
    public void TeardownTest()
    {
        try
        {
            selenium.Stop();
        }
        catch (Exception)
        {
            // Ignore errors if unable to close the browser
        }
        Assert.AreEqual("", verificationErrors.ToString());
    }

    [Test]
    public void TheNewTest()
    {
        selenium.Open("/Home");
        selenium.Type("currentYear", "2010");
        selenium.Type("birthYear", "1983");
        selenium.Click("//input[@value='Calculate age']");
        selenium.WaitForPageToLoad("10000");
        try
        {
            Assert.AreEqual("27", selenium.GetValue("result"));
        }
        catch (AssertionException e)
        {
            verificationErrors.Append(e.Message);
        }
    }
}

As you can see the tests are executed from NUnit.  While this sounds rather strange, it’s actually pretty useful – for example, adding your Selenium tests to a build process is straightforward.  Also, page elements are matched on id – if I change the layout of my page the tests won’t suddenly fail.

So now we simply start Selenium RC and run the tests.

Pretty neat.

Things I like about Selenium

The IDE.  This is probably the biggest advantage Selenium has over similar frameworks.  While this may not seem like a big deal at first (writing the code in this example wouldn’t be all that difficult) it does speed up the entire process and it allows testers to create the integration tests for you.  For this simple reason I would probably choose Selenium over any other web testing framework.

Selenium also allows you to run your tests in IE, Firefox, Safari and Opera. (Support for other browsers like Chrome is also possible depending on the browser’s security settings)  This is actually quite neat because you will immediately pick up any changes that works in some browsers but not in others.

Things I don’t like about Selenium

Take a closer look at the output from my unit tests – that one test took 17 seconds.  A ran the test a couple more times and the average seems to be around 7 seconds.  So the first run took quite a lot longer, but 7 seconds is still quite a long time.  If you’re looking at 500 or 600 Integration tests your integration tests are going to take at least an hour.  To be fair, we have to compare this to similar platforms and we can always use Selenium Grid to run tests in parallel, so you might find that this is actually the quickest framework available.  Just keep in mind that Integration tests take a long time to run.

The documentation.  While Selenium does have a pretty neat documentation section, there are a couple of elements missing.  For one thing, the code documentation on the client drivers (in this case, the .Net assembly) is nonexistent.  For example, look at the line where we wait for the page to load.

If you wanted to know what that method does or what the parameter is (seconds, milliseconds… a string?) you would probably need to make an educated guess or hope that someone blogged about it.  (I think it’s milliseconds, by the way)

The generated code is also not that great – this is probably nitpicking, but generating code that does a catchall exception is definitely not a great idea.

The installation can also be a whole lot simpler.  For starters, Selenium Core is available as a standalone and unnecessary download (you just need the IDE and RC).  If you’re anything like me you’re going to download it, fool around with it for a while and then get annoyed because you don’t need it.  Also, the RC download contains client drivers for all 6 languages supported by Selenium.  This means 99% of developers will be downloading the drivers for 5 languages they’re not going to use.  Lastly, no mention is made of how to actually start the RC Server.  Fitnesse also uses a java server but includes a .bat file for developers unfamiliar with java.  It’s not a big deal, but it tends to make your life just that little bit more difficult.  (You need to run java –jar selenium-server.jar to start it, by the way)

Summary

Selenium is one of the best, if not the best, web-application testing tools.  The ease with which tests can be created and run makes it stand out amongst similar frameworks.  The support for multiple browsers is extensive and tests can easily be integrated into an automated build process.

Advantages

• IDE takes care of test scripting
• Can be used by testers without any coding knowledge
• Supports multiple browsers
• Can run tests in parallel with Selenium Grid

Disadvantages

• Documentation is lacking in some areas
• Generated code is not that great

In my next post I will take a look at the Microsoft’s new Lightweight Test Automation Framework.

Share on: Twitter, Facebook, Google+ or use the Permalink