Happy Path Testing With Selenium RC Fu

Note: this was originally posted on the Pivotal Labs blog.

Selenium RC Fu is a fantastic system for testing Ruby On Rails applications. It is the blending of xUnit testing with selenium. Selenium is an amazing system that operates your browser as if a human were sitting there moving the mouse, pressing buttons and keys.

Selenium RC Fu is also a remarkable example of the power of open source. It’s Selenium remotely controlled by rails and ruby. You can learn more about it by viewing the slides for Full-stack webapp testing with Selenium and Rails presented by my colleagues Alex Chaffee and Brian Takita at the SDForum Silicon Valley Ruby Conference.

Now that you are excited about Selenium RC Fu, by law I must inform you that this wonderful testing tool comes with some costs. First, this is the daisy cutter of testing — problems will be detected, but it won’t be too specific about those problems. A failed selenium test will likely only tell you some expected text was not present on the page — you have to do some digging to discover the real problem.

It’s also slow. To be fair, a lot of software is running to do this testing.

So use selenium testing sparingly. A good strategy is to restrict selenium testing to “happy path” testing. These happy path tests become a compliment to other more focused and faster unit and integration tests.

Getting Started With Selenium RC Fu

Selenium RC Fu is hosted at rubyforge.org in the “pivotal.rb” project. It’s not well documented and a little hard to find. The primary documentation is the README File.

Step 1 is to add it to your project:


    script/plugin install svn://rubyforge.org/var/svn/pivotalrb/seleniumrc_fu/trunk

Building Your Tests

Create your tests in app/tests/selenium. The basic structure mirrors the other test types:



    require File.dirname(__FILE__) + "/selenium_helper"
    class HappyPathsTest < MyProject::SeleniumTestCase
    
      def test_nav_bar
          ...
      end
    
      def test_about
          ...
      end
    
      # more tests
    end

Note that HappyPathsTest extends something called MyProject::SeleniumTestCase. Selenium RC Fu provides a sample selenium helper file that suggests this convention for name spacing your tests.

For the happy paths tests I simply wanted to go to a page and know that the page loads. A pattern I learned from my colleague Shifra is:

  1. Pick something you know is on the target page, but not on the current page. Let’s call it ‘evidence’. Assert that evidence is not present on the current page.
  2. go to the target page
  3. assert the evidence is present on the target page.

This process insures that you are moving about as you expect. Here’s an example from my project:



      def test_tasks
        ...

        assert_element_not_present "xpath=//h1&91;text()='The Daily Planet']"
        click_and_wait "link=The Daily Planet"
        assert_element_present "xpath=//h1[text()='The Daily Planet']"

        ...    
      end

I repeated this simple pattern for every path.

Selenium Functions/Assertions/Commands

Here are the functions and assertions I used in my tests:

  • click_and_wait locator
  • go_back
  • wait_for_page_to_load
  • type locator, text
  • assert_text_present text
  • assert_text_not_present text
  • assert_equal value1, value2

Many of the functions require a ‘locator’ argument. This needs to identify a single element on the page. Often the name or id of an element is sufficient, but you may need to use an XPath. Check the docs for more information on element locators.

These functions were sufficient for me. Look for more in vendor/plugins/seleniumrc_fu/lib/seleniumrc_fu/selenium_dsl.rb.

Selenium RC Fu comes with some rake commands too (from rake -T selenium):



    rake selenium:restart_servant           # Stop and start the selenium servant (the server that launches browsers) on localhost
    rake selenium:run_server                # Run the selenium servant in the foreground
    rake selenium:server                    # Run the selenium remote-control server
    rake selenium:start_servant             # Start the selenium servant (the server that launches browsers) on localhost
    rake selenium:stop_servant              # Stop the selenium servant (the server that launches browsers) on localhost
    rake selenium:test                      # Run a selenium test file (default test/selenium/selenium_suite)
    rake selenium:test_with_server_started  # Run the selenium tests in Firefox

Running Your Tests

You can run your selenium tests individually or as part of a suite. Selenium RC Fu comes with a suite script in the vendor/plugins/seleniumnrc_fu/sample directory. Drop that in your test/selenium directory.

The selenium server must be running before running individual tests. You can start the server with the rake command rake selenium:server. Once the server is running you run your tests from the command line or via the suite.

You can also run the suite with the command rake selenium:test. This command is smart enough to start the server if it’s not already running. However, when the server is started by this mechanism its a little more work to stop it. You can stop the server with a simple ^c if it was started via the rake selenium:server command.

Make It So

That’s really all you need to build your tests but here are few more things that may help.

Site Diagram

A site diagram is handy for building your happy paths tests. There will be a lot of paths through the site and marking the paths on the diagram as you visit them is an easy way to track your progress. On the diagram you need to indicate both pages and the paths to those pages.

The later is really important. It’s the paths that you will be testing with your happy path selenium testing. Often there are buttons, links, or other controls such as check boxes that may touch the server and reload the current page, or just update the current page via ajax. Be sure to add these to your navigation diagram.

Here’s the diagram I used for my site.

site diagram

Selenium IDE

You can code your tests by hand using Selenium RC Fu functions but it’s easier if you use Selenium IDE. This is a Firefox plugin that records your activity. That activity is available in various selenium dialects and can be pasted into your tests. Unfortunately, it does not have output for Selenium RC Fu. Nonetheless I still found it useful. I simply clicked through my application while Selenium IDE was recording, pasted the resulting selenium commands into my tests, then converted those commands to equivalent Selenium RC Fu commands. For example:

Output from Selenium IDE:



    @selenium.click "link=My Account"
    @selenium.wait_for_page_to_load "30000"
    assert @selenium.is_text_present("Update Account")

Becomes:



    click_and_wait "link=My Account"
    assert_text_present "Update Account"

Test Data

Your selenium tests will be exercising your application just like standard unit or integration tests. These tests require data and there are a variety of techniques to make data available to your tests. Building your happy path tests is easier if you have fixture data and that fixture data is loaded in your development environment via rake db:fixtures:load. If you use fixture scenarious, it is handy to have a scenario for selenium testing, which you would load with something like rake db:scenario:load SCENARIO=selenium.

With your fixtures loaded in your development environment you can see exactly what is present during testing and the selenium IDE will record exactly what is played back by selenium.

XPath Checker

Usually Selenium IDE can assign a locator that simply works but there are times when it can’t or won’t. Using XPath Checker you can right click on an element and it will display an XPath to that element. You can also experiment with varitions and XPath Checker will list all of the elements on the page that can be identified by that XPath.

That’s it. Let me know how it goes.

Advertisements

11 thoughts on “Happy Path Testing With Selenium RC Fu

  1. You mention loading your fixtures in development environment, but without transactional fixtures available how do you ensure that the database is in a known state at the beginning of every test?

    Is it a case of reloading all (or perhaps some) of the fixtures between each test? Or is there something cleverer I’m missing?

  2. Henry,

    I believe that the normal, non-transactional fixture flow is 1) the fixture table is cleared and 2) the fixture data loaded, and this is done for each fixture specified for your tests. Transactional fixtures are an optimization to this process, since the fixture loading and the test are all done in a transaction, which is simply rolled back at the end of the test.

    I hope this helps.

  3. Henry,

    After rereading your comment and rereading the article, I think you may have been referring to the section where I suggest loading your fixture data into your dev environment. You asked about transactional fixtures and as I said in my last comment this did not cause a problem for me but it might for you.

    I’m not sure the article made the point clear enough so I’m going to try again. Your selenium tests will probably need data, and in the process of creating your selenium tests initially it is helpful to drive the application manually so that you can see exactly what the selenium test will see. The best way to do this is 1) use fixture data for your tests and 2) load that fixture data into your dev environment while creating your tests so that you can run your app and see it just as your selenium tests will see it.

  4. spent,

    Sorry for taking so long to reply, I got side-tracked by other issues.

    That’s cleared up one confusion, which is why you were loading your fixtures into your development environment. That makes good sense to be able to see what your tests will see when creating the tests.

    Looking back my main question is about the running of the tests after they’ve been written. Basically, I was wondering if there was any way around having to delete and reload the fixtures between each test. Clearly one possibility is using transactional fixtures as with the unit tests, though I was under the (possibly mistaken impression) that transactional fixtures as implemented unobtrusively by Rails wouldn’t work with Selenium RC-style tests. It’s a different process running the tests (rake selenium:test) to the one altering the database through the Rails app (the mongrel server), so any changes made by the mongrel process won’t be visible to the test task and the rollback by the test task won’t rollback any changes by the mongrel process.

    However, you say that transactional fixtures did not cause a problem for you. So does that mean, they just worked with Selenium tests without any need for additional configuration or setup?

  5. Henry,

    >…I was wondering if there was any way around having to delete
    >and reload the fixtures between each test…

    I don’t know of a way. The test scripts have full access to the rails data and simultaneously drive selenium to perform operations on a browser which is in turn talking to a web server. If things were transactional the test scripts would not be able to see the state of the data. The “Full-stack webapp testing with Selenium and Rails” talk mentioned at the beginning of the article probably covers this better.

    > …you say that transactional fixtures did not cause a problem for you.

    You are correct that transactional fixtures would break seleniumrc_fu’s operations. Transactional fixtures are turned off in the seleniumrc_fu test base class. My test suite now has a mix of tests, most of which use transactional fixtures, but the selenium tests do not.

    These tests are slow but really useful. At Pivotal Labs we use continuous integration, which partially takes the sting out of long running tests. You can make a change and run the tests that verify the change and check in. The CI box will run the full test suite, including the selenium tests on multiple browsers (firefox, IE).

  6. Thanks, that clears everything up. There’s no doubt that we’re going to include selenium tests, I was just hoping that I was being dense and there was be an easy way to speed up these tests through transactions – definitely too good to be true.

  7. I tried following your instructions, and the rake task complained like this:

    ** Invoke selenium:test_with_server_started (first_time)
    ** Execute selenium:test_with_server_started
    rake aborted!
    Since Selenium spawns an internal app server, we need ActiveRecord to be multi-threaded. Please set ‘ActiveRecord::Base.allow_concurrency = true’ in your environment file (e.g. test.rb).

    Sure enough, I got past the error by adding the suggested line to config/environments/test.rb

    The next problem that I ran into was that the selenium server was looking for a binary called “firefox-bin” in my path. I’m running firefox 3 on linux, and it’s binary is just called “firefox”. I fixed the problem by making a symbolic link: /usr/local/bin/firefox-bin -> /usr/lib/firefox-3.0.1/firefox

    After that my test ran fine.

    Has anybody found a good way to configure selenium-rc_fu to use specified firefox profiles? I would like to have some tests that check how my app behaves in a browser with javascript enabled, and other tests that verify that the app gracefully degrades without javascript support (e.g. use a firefox profile with javascript disabled.)

  8. Hi

    I am currently working on QTP for some GE projects and as per the req we are moving from qtp to Selenium RC
    As i am a new baby in selenium kindly tell me each and every step staring from download selenium files from path till the complete script running part using RUBY client(As we decided to use ruby client)

    Kindly give me the examples like where to keep the selenium server files ruby files and our script file and when and how to run the server and when and how to run the ruby scripts

    We are trying this from last 10 days and found no success
    Your reply will be appreciated

    Regards
    Test

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s