Cucumber And Its Role In Bridging The Gap Between Expectation And Reality Of A Software Solution In DevOps

With a specific end goal to improve favorable position of the software testing, companies are these days stepping forward. While unit tests have been used to increase quality of the code the main issue is whether the concept for the software has been correctly understood by the development team, because if the requirements are misunderstood then even having unit tested code will not be a measure of the quality of the software. Behavior Driven Development is a way to alleviate this issue, the stakeholder cannot always be technically proficient, the next best thing would be for them to actively participate in the quality assurance from the grass root level. Cucumber is one of the leading tools supporting Behavior Driven Development.

Introduction

Any software begins with an idea that is then translated into a working application. The better the idea is understood the better the application will align with the idea. If the person who has the idea can design the application and implement it, that application might be the closest to align with the idea. While this is not the case for majority of the applications the quality aspect of software cannot depend on the developer being the idea creator.

The next best thing would be for the development team to fully understand the idea or the requirements of the person who has the idea hereafter the client. This can be done through communication and collaboration as well as frequent demonstrations of the application during its development. Now how well the software aligns with the client’s requirements depends on the development team understanding of the requirements. With large teams and large software projects the chain of information can be longer and the chances of misrepresentation of an idea is that much higher. Requirements can and may be interpreted in different ways if detailed description is scarce.

Using the concepts of “Test Driven Development”, if the unit tests are written first and based on this the code is written we can have a higher level of quality as the code is written with the unit test as base. The unit test is however again subject to the developer’s understanding of the requirements. What if we were to write unit tests based on the simple worded requirements that can well reduce the chance of misinterpretation. This is the solution “Behavior Driven Development” provides, the client or a business analyst can write the requirements in simple English and the developer can write code wired up to these English statements.

Cucumber is a testing framework that helps to bridge the gap between software developers and business managers. Tests are written in plain language based on the behavior-driven development (BDD) style of Given, When, Then, which any layperson can understand. Test cases are then placed into feature files that cover one or more test scenarios. Cucumber makes it very easy to handle cases of different business scenarios with different input data and different results based on that input data. The scenario is defined with Scenario Outline. Then data is fed to this scenario with Examples table where variables are defined with concrete values.

Testing with Cucumber

Cucumber tests written in feature files are translated into programming code and executed. The BDD Given, When, Then syntax is designed to be intuitive. Consider the syntax elements:

• Given provides context for the test scenario about to be executed, such as the point in your application that the test occurs as well as any prerequisite data.
• When specifies the set of actions that triggers the test, such as user or subsystem actions.
• Then specifies the expected result of the test.

Cucumber in itself does not run the tests but provides an interface from simple instruction in the feature files being converted into code that can be executed and verified. Web based application functionality is usually tested using Selenium. Cucumber integrated well with Selenium for testing the functionality of a web application.

Cucumber reads the feature files in a specified location and executes the specified tests.

1. Feature Files

The feature files give the basic idea of the features in plain English like language Gherkin that is used by cucumber. The feature files themselves do not provide the functionality for the tests, they just give a structure on how or what needs to be run on the test. For specifying what the lines on the feature files translate to we provide the steps to be performed translated to a programming language. The feature files will generally be used by a non-technical person like the client or the business analyst while the step definition files will be done by the programmer or automation engineer. The step definition needs to be defined for each line of the feature file in a generic way so that it may be reused. For example, say we have a step for logging into the client’s webpage and clicking a button, the programmer will write the step definition in such a way that any other feature file can also use the same step definition while needing to login to a page. Each method is annotated with a Given, When, or Then, which contains a regular expression matching the lines in the feature file. This regular expression is what enables reusability of the steps in multiple features.

2. Selenium Browser Testing

Define abbreviations and acronyms the first time they are used in the text, even after they have been defined in the abstract. Abbreviations such as IEEE, SI, MKS, CGS, sc, dc, and rms do not have to be defined. Do not use abbreviations in the title or heads unless they are unavoidable. As specified before Cucumber is just an interface between a non-technical user and the test executing mechanism. The test execution may be done in a variety of ways like NUnit, MSTest etc in case of unit tests. In the case of web-based applications while unit testing is important automated browser tests enable us to fully ensure the E2E quality of the product. Selenium is used for this web application testing. The client writing the feature files need not be aware of any language or technology, they can specify that a page needs to be navigated to. This will be internally coded in a generic way by the programmer in Selenium in the step definition.

3. Cucumber and the DevOps Pipeline

The DevOps pipeline is a is a software strategy that enables organizations to deliver new features to users as fast and efficiently as possible. The core idea of CD is to create a repeatable, reliable and incrementally improving process for taking software from concept to customer. The goal of Continuous Delivery is to enable a constant flow of changes into production via an automated software production line. After build of the application the next important step is to test the application. In a pipeline the Tests need to be automated, repeatable and reliable. Cucumber brings in an extra layer of understanding to the client as they can see the results of their features when its run by the pipeline. Cucumber generates reports for the executed tests providing a graphical and easily understandable way to judge the quality of the application. Quality gates may be set such that the tests need to be run successfully for the pipeline to move to its next stage. This ensures that functionality of an application is ensured in an automated way by the pipeline. Jenkins is an open source DevOps tool used for orchestrating the DevOps delivery process. Cucumber can be easily integrated to the Jenkins pipeline to ensure quality of the application.

Working with Cucumber

We start with an application without any end to end test suites. The application consists of a web-based frontend written in PHP and a MySQL database to store and retrieve data for the web page. Currently without tests it would be difficult to ensure quality of the application without deploying it into an actual server. Manual quality assurance while effective can be very time consuming as well as repetitive. Bringing in automated end to end testing can alleviate this issue while also making the solution DevOps pipeline friendly. What Cucumber brings to the table is the ease with which a non-technical user can understand the tests as well as provide a way to participate in the coding process which otherwise be confined to technical users alone.

1. Setting up the application

The web application comprising of PHP frontend and MySQL backend is containerised. This makes the process portable and repeatable. The images are built using Multi Stage DockerFiles. Initial step would be to clone the repository from GitHub. These files are then copied on to the respective base files for the images. For the PHP web application, the base image used is php:7. 2-apache For the MySQL database, the base image used is mysql:5. 6 Now the PHP website is hosted in Apache which is a part of the base image. The MySQL container will be seeded with data on being built, this sql file will also be copied in from the GitHub repository.

2. Setting up the Test Suite

As this is a web application we will be using Selenium to implement the browser testing functionality. We will again be leveraging containerisation for this purpose. Instead of a single selenium instance tied directly to the test we would be using selenium grid so that we are not confined to a single instance but have a scalable collection of selenium instances. We would be running three different kinds of selenium containers. First we will be running the selenium hub image specifically selenium/hub:3. 14. 0-francium. This acts as the point of contact for the tests requiring the selenium instance. The port 4444 will be exposed for this. Now the selenium hub in itself will not suffice for selenium browser tests. For this we would require selenium nodes to be connected to the hub. We can connect any number of selenium nodes to the hub so as to scale the performance of the system as well as parallel execution of the tests. We will be connecting two selenium nodes, selenium/node-chrome-debug:3. 14. 0-francium and selenium/node-firefox-debug:3. 14. 0-francium. These nodes will have the chrome and firefox browsers respectively. This special type of debug container is being used as it provides a way to view the browser inside the container using a VNC viewer. For this purpose the selenium nodes will be exposing their 5900 ports, this will be mapped to a separate port on the host docker machine.

3. Cucumber Implementation

We will be using the. NET flavor of Cucumber called SpecFlow in this demonstration. The IDE used for this purpose will be Visual Studio 2017 Community Edition. Setting up the Cucumber test solution is really straight forward. Once all the required dependencies are installed we will be creating a Cucumber Feature File for the login functionality of the web application. The Feature file will be run using some generic page navigation functionality written using Selenium. As we are using Selenium Hub in the application we would be using a RemoteWebDriver instance to initialise the selenium webdriver instance. The generic step definition is written for page navigation, button click, text population and verifying required page. These will serve as the backbone for scenarios written in the feature file.

Based on these steps a number of scenarios can now be written in plain English. The steps should follow a pattern for it to be bound to the predefined step definition, this can be easily understood. We can have multiple browser tests using a Cucumber feature called tags, based on the tags we can decide on which browser to load. The BeforeScenario is a Cucumber feature called hook which lets us run initialisation code that needs to run before the scenarios are executed. In this case we use this hook to initialise the driver instance for the browser tests. The tags can be used to target a specific hook, in our case we use tags to execute tests in specific browser instances connected to Selenium Hub. Likewise AfterScenario is hook used to do any cleanup work after the scenario runs. There are three separate scenarios that can be checked for login functionality.

1. A user clicks on login button should be redirected to login page. For the same scenario we can test both Chrome and Firefox browsers. This differentiation is achieved using tags.
2. A valid user who enters valid credentials should be redirected to their respective profile page. This again will be checked in multiple browsers.
3. A non registered user when submitting credentials must be redirected to the registration page. This will also be checked in both chrome and firefox.

All these features can be easily understood by a non-technical person who can also easily add more scenarios easily. For executing these the test will connect to the Selenium Hub and based on the required capability, that particular Selenium hub will execute the feature scenario requirements.

Conclusion

Setting up an automated framework for web application testing is a challenge, and also demonstrably worthwhile in the long run. From this work we are able to establish how Cucumber can allow non-technical users to understand and involve actively in the quality of the product. The advent of newer technologies like docker helps in strengthening the role of Cucumber in the development life cycle with the ease with which containers and entire test suites can be spun up giving us immediate and observable feedback. Involving the client or other stakeholder to actively participate in the development process allows us to deliver better quality quickly. Furthermore, Cucumber allows easy integration to the DevOps pipeline thus giving the customer more feedback as well as a framework for reliable regression.