Comparative Analysis Of Various Tools For Automation Testing Of Angularjs

Abstract

AngularJS is the new Javascript framework, being widely used across web browsers and also having a great expression power is paired with a shortcoming i. e. lack of compiler optimization. Thereby, it is recommended strongly that every application written in JavaScript, irrespective of the used framework, should include tests that validate both its behavior and performance. This paper provides a comparitive analysis of the various testing tools available for testing the AngularJS applications mainly focusing on the two most popular tools namely Karma and Protactor.

Keywords— Testing, Web Automated testing, Unit testing, End-to-end testing, Karma, Protractor

Introduction

In Test-Driven Development (TDD) Automated Testing is a key element. In Test-driven development we write test cases for each unit before the actual code is written. These unit tests are run after the code is written, and based on the results of these tests, code can be restructured (if necessary) or new tests can be added and executed. Few major types of testing are: Unit testing – Testing units of a software product. Functional testing – Software testing is performed in order to detect actual performance of an application’s functional requirements.

Various web frameworks such as AngularJS, ReactJS, KnockoutJS, PolymerJS, used for dynamic development of web application have been recently introduced. The key points of their popularity are concepts like data-binding, templating, routing, features of single-application pages, MVC and etc. Regardless of speed at which they can be learned and implemented, the final product needs to pass a certain level of quality assurance, before it can be delivered to the end user. By providing continuous integration of tests with source code, this challenge can be overcome. Few web frameworks for testing applications are Jasmine, Mocha and Coffee. Because of its readability and integration across testing tools, Jasmine was chosen. Jasmine tests consist of suites and specs. Suites consist of related tests for application and each test is called a spec. It also contains matchers which compare results of test, called actual values with some expected values. This makes documentation easy to generate after test execution because brief string descriptions inside specs and suites explain the main functionality of tested module.

Every AngularJS application contains the MVC Architecture. MVC stands for Model, View and Controller. The three components of MVC are - Model, View and Controller. Model: It represents the shape of data and it’s business logic. View: View is a user interface. View display data using model to the user and also enables them to modify the data. Controller: Controller handles the user requests.

Description of few testing methodologies in parallel with corresponding tool to test AngularJS application will be introduced in the following chapters. Conclusion provides a detailed result analysis.

Angularjs Testing And Frameworks

AngularJS is basically a framework used for organization of JavaScript code in highly interactive web and mobile applications. Other popular frameworks include jQuery, ReactJS, Vue, and Ember, and all of them serve the same purpose. For having a modern-looking, dynamic web application with a user experience similar to a native mobile application these languages are used. From a testing perspective it means that apart from functionality, which may vary based on the app, a lot of effort for testing micro functionalities of front-end interactions. These behaviors vary based on the version of the web browser which are not easily detectable with GUI automation tools. While the HTML DOM stays the same,Visual layout and alignment may vary, so tools like Selenium are quite unsuccessful at detecting minor or major deviations. Also, majority of business logic can be tested beneath the UI or before it becomes available as modern web applications use Web Services architecture. This approach has a crucial requirement i. e. the Testers should be able to operate API automation tools and have a certain proficiency in programming. However, this is not specific to Angular, modern applications are designed in responsive fashion—the application’s visual layout changes based on screen size, platform, and orientation of mobile device. Combinations of different sets of grids, frames, images, and controls also need to be tested as Implementation of responsive design requires them. In our study, we’ve compared the tools that are the topmost trending tools for testing the AngularJS applications. Few key points of them are as mentioned below.

1. Protractor

Protractor is an end-to-end test framework used for Angular and AngularJS applications. Protractor runs tests against application running in a real browser, interacting with it as a user. Protractor is built over WebDriverJS, which uses native events and browser-specific drivers to interact with application as a user. It works as a Solution integrator combining powerful technologies like Selenium, Jasmine, Web driver, etc. It supports behavior-driven development frameworks like Jasmine, Mocha, Cucumber etc. It uses Selenium grid to run multiple browsers at once.

While working with protractor, following things are important;Protractor uses WebDriverJS while it uses the JavaScript as bindings for the Selenium WebDriver API. Commands used by WebDriver are asynchronous. The test scripts send commands to the Selenium Server, which communicates with the browser driver in turn. Advantages: There is no need to add waits and sleeps, as the webpage finishes pending tasks, executes the next steps of the testing process by automatically connecting with the AngularJS application. It is easy to set up page objects. Protractor does not perform WebDriver commands till an action is needed as they set up page objects so that tests can operate page elements without moving the HTML. Protractor supports AngularJS.

2. Karma - The main goal of the Angular

JS team for development of Karma is to bring an efficient and productive testing environment to the developers. There is no need to set up loads of configurations in the environment. Rather it’s a place where developers can just write the code and get instant feedback from their tests. Along with the property of playing nicely with Angular, it also provides flexibility for you to tailor the tests to your workflow. This includes the option to test your code on various browsers and devices such as phones, tablets, etc. In Karma we can also replace Jasmine with other testing frameworks such as Mocha and QUnit or also integrate with various continuous integration services like Jenkins, TravisCI, or CircleCI. Some features of Karma are: Control the whole workflow from the command line or your IDE - just save a file and Karma will run all the tests. Describe your tests with Jasmine, Mocha, QUnit, or write a simple adapter for any framework you like. Test your code on real browsers and real devices such as phones, tablets or on a headless PhantomJS instance. Easy debugging directly from your IDE via WebStorm or Google Chrome. Developed for and maintained by the open source community at GitHub. Simple integration with Jenkins, Travis or Semaphore.

3. Jest - Jest is an open Java

Script testing library provided by Facebook. Features of Jest: Easy to Set Up: Jest is pretty simple to install on its own. Provides Lightning Fast Speed: Jest is very fast. You can save significant time from your test runs when your tests are CPU bound. One in all package: Jest comes with built-in matchers, spies, and its own extensive mocking library. It used to be based on Jasmine, so it has all of Jasmine's goodness. Has Mocks: Mocking is an incredibly important part of unit testing. It allows you to replace irrelevant dependencies that may be slow and even control time for code that relies on timing. Snapshot Testing: Snapshot testing lets you capture a string that represents your rendered component and store it in a file which can be compared later to ensure that the UI remains unchanged. Watch Mode: It runs the tests automatically whenever you change the code. Zero configuration: Jest does not require much of configuration beforehand.

Suitable Tools For Unit And End-to-end Testing

The part of the application to which access is granted is the main difference between unit and end-to-end testing methodologies. Unit test can access model and controller (Fig 1. ). Unit Comparison of AngularJS framework testing tools test tools can inject controller, services, as well as filters. Hence, a developer can test every module of the application independently. End-to-end test can only access the view. The main functionalities can be tested using HTML template of an application, for example opening a new view when a button is clicked. The application under test must be complete and should be running on a server for end-to-end application testing. Its advantage is that it conducts tests in a manner similar to manual testing, where a human operator would be testing the application.

Results

It is always advised to reconsider all options of the tools as well as what each of testing strategies offer before any testing of an application is conducted. Unit testing, conducted with Karma is considered more agile. With the use of breakpoints debugging unit test becomes easy. Karma offers an integrated Webdriver alongwith code coverage. Most of browsers support unit tests and offer autowatch functions to observe source code changes (Table 1. ). The other side is that unit tests cannot be run in multiple browsers simultaneously. Also, these unit tests cannot test HTTP GET and HTTP POST requests to the server. It provides support to inject and mock Angular services into unit tests.

End-to-end testing is comparatively less flexible and gives fewer opportunities to check for an application fault than a unit test (Table 2. ). Currently it is not supported by all browsers, however as an advantage it can be run simultaneously in more browsers. Although it doesn’t have an integrated autowatch function, autowatch can be integrated in Javascript task runner called Grunt. End-to-end tests also require separate running instances of Selenium Webdriver during testing. Because of long error messages and the separation between the browser and the process running the test debugging of a test can be quite complicated. One solution can be capturing screenshots of the browser during tests to ease debugging.

Conclusion

For creating a maintainable and reusable code testing of applications is a key component. For understanding functionality and writing documentation using descriptive frameworks like Jasmine and Mocha to write a test is a good start. An untested software creates various problems such as: complicated DSL structures, duplicated code, and unreadable code. There are multiple advantages of tests from easily testing across more browsers, to no need to constantly refresh pages and use primitive debugging techniques. It is advisable to write down all expected functionalities, as well as user-specific requirements before performing testing on an application thereby writing more efficient tests. Depending on how large the application’s module is, it is always advisable to start from unit tests. These prove to be good indicators of how well internal structures of your application are working and show whether or not the application satisfies main functionality. Unit tests can be a good indicator of how well is the application is optimized, as well as provide code coverage which is crucial while writing smaller and better modules of an application. On the other hand, end-to-end test give an overall picture of application. They are important in testing a final product, the way application with all connected modules and units works. However, they are not so essential in determining application efficiency. The main advantage of end-to-end testing is that they can help integrate more modules tested with unit tests. Eventually the application can be fully functional and optimized regardless of its size and complexity. Considering this we can give a conclusion that Karma is a preferable suit for Unit testing of any application written in AngularJS. Also, end-to-end testing can be well done using Protractor as a automation testing tool.