User Acceptance Testing with Selenium and Cucumber

In the last implementation project I participated in we applied the Behavior Driven Development approach where the user stories are defined in Given-When-Then style. In this article I want to describe how I combined Cucumber with Selenium in order to automate our User-Acceptance Tests using Behavior Driven Development.

For running automated BDD tests there are some free frameworks available (a brief comparison of BDD frameworks). We decided to go for Cucumber because it served all our requirements, has a good documentation with good examples and is pretty easy to get it up and running

Cucumber JUnit Test

The following listing shows a simple example that is practically the archetype for all our Cucumber tests.

@RunWith(Cucumber.class)
@CucumberOptions(
  features = { "classpath:features/example/" },
  glue = { "my.project.uat." },
  tags = { "@Example" })
public class ExampleCucumberTest {
  //empty
}

The annotations of the example in detail are:

• @RunWith declares the TestRunner for this test, which is the Cucumber class. The test won't run without it.
• @CucumberOptions define various options for this tests. The options are optional but are quite helpful in controlling the behavior of the test
• features: declares a path were the BDD feature files (text files) are found. The example points to a location in the classpath. All feature files (.feature extension) below that location are considered. Multiple locations can be defined as an array.
• glue: defines the packages where Steps and Hooks are located. Steps and Hooks contain the actual code for the tests. Multiple packages can be defined as an array.
• tags: defines which stories should be executed. If you omit this option, all Stories are executed, otherwise only those that have one of the tags set will be run.

Of course there are more options available (see Cucumber apidoc)  but these are the options I use most. As soon as you have your first story written, you can start right away with this simple test.

BDD Feature 

The following exaple story is taken from the Cucumber documentation

@Example
Feature: Search courses
  Courses should be searchable by topic
  Search results should provide the course code

Scenario: Search by topic
    Given there are 240 courses which do not have the topic "biology"
    And there are 2 courses, A001 and B205, that each have "biology" as one of the topics
    When I search for "biology"
    Then I should see the following courses:
      | Course code |
      | A001        |
      | B205        |

With that feature in the right location you can run the above test with JUnit. Of course it will not be successful. Actually, with the default settings it will ignore all the steps unless you use the @CucumberOption(strict=true) which is recommended when you run the test as part of a quality gate.
The Cucumber documentation provides some good descriptions on Features and their syntax. You can define Backgrounds that are executed for each scenario of the feature (similar to JUnit 4 @BeforeClass) or Scenario Outlines to run the feature against a set of data. It is even possible to write your BDDs in different languages. Therefore you have to start the feature with the line following line and have all the keywords in the according language.

#language: de
Funktionalität: ...

But you have to be careful with the encoding of the Feature files and special characters. It's best to use UTF-8 as default encoding. A complete list of the keywords in other languages can be found in the Cucumber apidoc.

Cucumber Steps

When the test for the feature is run and the steps or part of it are not yet implemented, it will produce an output like this:

You can implement missing steps with the snippets below:

@Given("^there are (\\d+) courses which do not have the topic \"([^\"]*)\"$")
public void there_are_courses_which_do_not_have_the_topic(int arg1, String arg2) throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}

@Given("^there are (\\d+) courses, A(\\d+) and B(\\d+), that each have \"([^\"]*)\" as one of the topics$")
public void there_are_courses_A_and_B_that_each_have_as_one_of_the_topics(int arg1, int arg2, int arg3, String arg4) throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}
...

The test prints out skeletons for unimplemented steps. What you do now is to create a new step which is a simple Java class and put in one of the packages defined in the glue. Copy the skeletons into the class and implement it.
So the steps are basically what is executed for each line. It is possible to pass parameters to the steps that are extracted and converted so that steps can be reused with different values. Its also possible to define entire tables as input.

Hooks

Hooks are basically the same as steps but fulfill a similar role like the JUnit @Before and @After annotated methods, the even use the similar annotations (actually, the are named the same but are in a different package). You can trigger certain hooks using tags like shown in the following example:

@WithFirefox
Scenario: Response times with 10 users
...

and the according hook in Java will be

@Before("@WithFirefox")
public class BrowserHook {
 ...
  public void setupScenario_Firefox() {
   ...
  }
 ...

Dependencies between Hooks and Steps

In order to reuse existing code or to access the state of a particular Hook or Step instance you can create a dependency between the classes by defining a constructor that accepts a particular Hook or Step. The Cucumber JUnit runner will create instances of the according classes and inject them in classes that are dependent.

public class MyBrowserSteps {

  private BrowserHook browserHook;

  public MyBrowserSteps(final BrowserHooks browserHook) {
    this.browserHook = browserHook;
  }

The same applies to Steps so you can make one set of steps dependent on other steps.

Selenium Steps

So far I only described how to write any test with Cucumber, but for User Acceptance Testing you might want to test the actual solution. For web application that is the deployed application that is accessed by a browser. For browser automation the Selenium framework is widely known and framework of choice for most cases. It provides a recording tool (a plugin to Firefox) to record user interactions with the browser. It provides a model to access elements of the website using Java and various methods of locating elements in the browser.
For automated user-acceptance tests with Selenium and Cucumber the basic approach would be to

1. Record actions with the Selenium Recorder
2. Copy them to Steps classes that match your BDD
3. Define assertions in Then step implementations

A step implemented with Selenium will look like in the following example

@When("^I push the button$")
public void i_push_the_button() throws Throwable {
  driver.findElement(By.cssSelector("div.v-select-button")).click();
}

In order to reuse steps for different browsers, I used the BrowserHook shown in on of the example on above and set-up the browser using a specific tag for each browser. The driver is first initialized upon the first call to getDriver(). In the step itself I retrieve the driver from the BrowserHook that got injected. The BrowserHook may be implemented like this

public final class BrowserHooks {
  private enum DriverType {
    headless,
    firefox,
    ie,
    chrome, ;
  }
  
  private WebDriver driver;

  public WebDriver getDriver() {
    if (driver == null) {
      switch (driverType) {
 case ie:
   driver = new InternetExplorerDriver();
   break;
        case firefox:
          driver = new FirefoxDriver();
          break;
        ...
    }
    return driver;
  }

  @Before("@WithFirefox")
  public void setupScenario_Firefox() {
    driverType = DriverType.firefox;
  }
  
  @Before("@WithIE")
  public void setupScenario_InternetExplorer() {
    driverType = DriverType.ie;
  }
  ...
}

And the step definition that uses it may look like

public class MyBrowserSteps {

  private BrowserHook browserHook;

  public MyBrowserSteps(final BrowserHooks browserHook) {
    this.browserHook = browserHook;
  }

  @When("^I push the button$")
  public void i_push_the_button() throws Throwable {
    this.browserHook.getDriver().findElement(By.cssSelector("div.v-select-button")).click();
  }
...

Aggregate Steps

One of the big advantages of a BDD framework like Cucumber is, that you can define steps that aggregate multiple steps. A good example for this is the Login Story. Although this is a point of typical discussions whether "Login User" is a valid Use Case or User Story (with regards to its business value) the requirement to allow a user to login does exists and its parameters need to be defined (whether it is via Single Sign On, Smartcard, Username/Password, Two-Factor or whatever else).
So let's assume you define a login user story such as

 Given the login screen is being displayed
 When I enter my username "xxx" and my password "yyy"
  And I push the login button
 Then I see the main screen of the application
  And I see my name being displayed in the user info box

Now you don't want to describe all these steps over and over again because the rest of the application under test requires a logged in user. So you could begin the other stories with

• When the user "xxx" is logged in

Or even better by using a hook/tag before the story like

• @Authenticated

Now, what you do in your code is to define a dependency to the steps class that contains the login step definitions and invoke each of them in the correct order either in a hook definition or in a step definition. The advantage of a hook is that you could combine it with other hooks to set up the test user or even persona.

public class LoginSteps {

  private BrowserHook browserHook;

  public LoginSteps (final BrowserHooks browserHook) {
    this.browserHook = browserHook;
  }


  @Given("^the login screen is being displayed$")
  public the_login_screen_is_being_displayed() {
    this.browserHook.getDriver().get(baseURL);
  }
  @When("^I enter my username \"([^\"]*)\" and my password \"([^\"]*)\"$")
  public void I_enter_my_username_and_my_password(String arg1, String arg2) throws Throwable {
    //with Selenium, put in the values in the login form
  }
  @When("^I push the login button$")
  public void I_push_the_login_button() throws Throwable {
    // with Selenium, locate the submit/login button and click it
  }
  ...
}

public class LoginHook {

  private LoginSteps loginSteps;
  private String testUser;
  private String testPassword;

  public LoginHook (final LoginSteps loginSteps) {
    this.loginSteps= loginSteps;
   
  }

  @Before(value="@PersonaXY", order=1)
  public void selectPersonaXY() {
     this.testUser = ...;
     this.testPassword = ...;
  }

  @Before(value="@Authenticated", order=2)
  public void login() {
    this.loginSteps.the_login_screen_is_being_displayed();
    this.loginSteps.I_enter_my_username_and_my_password(testUser, testUserPassword);
    this.loginSteps.I_push_the_login_button();
    ...
  }
}

And how it is used in a story

 @Authenticated @PersonaXY
 Given I see the meaningful screen
 When I do something purposeful
 Then I get a sensible result

 Conclusion

In this article I gave a brief introduction into Cucumber and how to write testcases with it. I showed how to define steps with Selenium to create meaningful, browser-based user acceptance testing and I showed how to combine thereby reuse steps and hook to create a rich user acceptance testing suite.