Why data driven testing?
Tasks in testing services such as quality assurance tasks sometimes require you to examine how different “classes” of data will perform in the same test procedure. A perfect example to explain this is a login to website. The classic way to test this case is to give a valid email and valid password and click on submit. The expected result is you to login into the system under test. But there are several negative testing scenarios which needs to be covered as well.
A) Valid email, invalid password = Error message 1
B) Valid email, blank password=Error message 2
C) Blank email, blank password= Error message 3
D) Unregistered email, valid password= Error message 4
The above scenarios would require you to create 4 different test cases and test each one of these individually. Or since they are effectively doing the same thing, meaning, supplying email and password and clicking on the submit button and checking for the error message, you could loop through a series of data values in the same test case and keep track of which one is passes or fails.
Another scenario where data driven tests can help would be data setup. Automation doesn’t always have to test things. We could use the same concept of iterating through different data rows to do the set up. For e.g. When the development team deploys new instance of application under test, Jenkins could be used to trigger a particular scenario that will do a data driven process to register 100 new users into the system under test from an external file like CSV, JSON or Excel. So whether you are doing a testing scenario like Login or verifying the different kind of people that can register or whether you are doing data set up, understanding how robot framework can load up multiple rows of data from the file and process them through 1 test case is a great skill to have.
Here is a typical example of a test automation project that has three separate test cases testing the negative condition
You can see the first test case, unregistered user should see correct error message at login. In the next test case, a registered user logins with invalid password should show correct error message and finally attempting to Login with blank email and password should show correct error message. Here you could see that the test cases does exactly the same test steps, but with different data in each one.
The repeated test steps here are:
1) Navigate to Sign-in page
2) Attempt a login with some credentials
3) Verify that error message displayed in login page, matches with what we would expect when we provide an input data.
Though we could test these scenarios as 3 different cases, a better approach would be create 1 test case that would say, negative login scenario should produce correct error messages and then would iterate through the different sets of the data for the same test step in 1 test case. If one of them failed, we wouldn’t exit the test case. We would simply keep track of the test case and then move on to next row of data.
Here is a sample program on how you could achieve this
Execute the *.robot program and the HTML results can be viewed as below.
Data driven tests in robot allows you to view each test data as a separate test case in the log files.
So typically, the built-in method for getting data driven results in robot framework are as follows
- Create a keyword with the common steps
- Either in the Tests file or the keyword file
- It has [Arguments] to catch test data
- Add [Test Template] to the test case
- Associates the keyword having common steps
- Add data rows to the test case
- These will be passed into keyword[Arguments]
