Behaviour Driven Development - Working with Data

Friday, 23 November 2018

In this article, we are going to walk through one of the challenges facing adopters of Behaviour Driven Development: how to manage data used in scenarios.

Let’s take an example:

Scenario: Logging into the system
Given I am on the login page
Then I fill in username with “Bob”
And I fill in password with “password123”
And I click “Login”
Then I should see “Welcome Bob”

This scenario specifies what should happen when Bob tries to log in to a system. All well and good. But we will probably want to test with more than one user including various invalid data. Won’t our feature file get a bit big and repetitive? Gherkin provides a mechanism for avoiding this bloat and duplication: the scenario outline.

Scenario outlines

Here the data is fed into our scenario from a pipe separated list of values.

Scenario Outline: Logging into the system

So we now can use the same scenario to test with valid or invalid account details. As good as this is, it does rely on the user details existing in the system under test. So what do we do when we want to test against a different environment which may have different user details available?

Testing against different environments

To execute our scenario against a different data we could copy the feature file and change the data in the table. As a pragmatic solution, this may be all that is required. Now we have two (or more) feature files each set with data for different environments. However, there are other reasons for wanting to duplicate a scenario.

Testing with different scope

For some points in the application lifecycle we only need a brief check – a sanity test that whatever changes have been made, the thing hasn’t fallen over. For other phases, we may need a more rigorous data validation test or even an exhaustive test. By taking the same scenario outline and changing the amount of data in the table, we can test with different scope. We can then choose which scenario outline to run using tagging.

However, we now have another problem. What happens when the system under test changes?

When the system under test changes

Let’s say that the login process needs to change. Imagine the security team are unhappy with the present form and want users to also provide an authentication code.

Using a BDD approach we want the business to specify the new functionality in a feature file. So which bit of which file should the business user work on?

Our pragmatic workarounds for testing in different environments and with different scope are starting to create a mess. Messes don’t get maintained – they get binned.

Keeping things DRY

As we have seen, testing for different environments and different scopes can be achieved by copying the scenarios but we have created a maintenance problem. We are also breaking a golden rule in development: Don’t Repeat Yourself.

In an effort to keep our scenarios clean, it is tempting to take the data out of the feature files altogether and then introduce it using a data loading step and modifying our step definitions to use this data. So we may end up with something like this:

Scenario: Logging into the system
Given I am on the login page
And I have got the login details from the login data file
Then I fill in username with the value from the login data file
And I fill in password with the value from the login data file
And I click “Login”
Then I should see the value from the login data file

Getting around this by developing technical solutions for extracting data at runtime creates a different problem. Business users no longer see or work on the data being used to run the test and we lose one of the key benefits of BDD.

To get around this we can use tools like mandolin which extracts tables into separate data files and then different data can be merged and used at will.

But whatever approach you take, make sure you plan data into your development.

Find out more

If you would like to know more about making Behaviour Driven Development work for you, please contact us.