Drawing on the experience in test automation of our QA consultants and engineers, ScienceSoft has compiled a list of popular tools for building a test automation toolkit in 2020. The list includes various categories of automated testing tools and tools related to all other aspects of test automation. We have employed them in our practice or see as particularly promising and winning the trust of the test automation community.
In testing via UI, the automation testing tools can be categorized into the following fields:
UI automation tools
These tools help to check UI components and an application’s end-to-end functionality by mimicking the actions of a user in a browser – clicking a button, switching a page, entering symbols into a field and more.
- Selenium WebDriver 3.0
In the pool of automated testing tools for web testing via UI, Selenium is the core one. It is an open-source and platform-independent tool, which supports test automation for all major web browsers. Since Selenium is basically a set of APIs, it provides a flexible and scalable way to test web applications. However, Selenium requires good programming skills to install, integrate and work with other tools.
UFT is the most popular tool among commercial alternatives to Selenium. It is a robust, user-friendly test automation tool by Micro Focus (now, part of HPE Software). UFT is easy to install, has rather good in-built reporting, and in addition to web testing, supports mobile, desktop, API/web service and database testing. At the same time, UFT-based test automation has several important limitations. It runs in Windows environments only and uses the Visual Basic language for scripting.
Other options: Protractor, Katalon, Ranorex, Robot Framework, TestComplete, Cypress, Nightwatch.js, Puppeteer, Serenity.
Visual validation tools
Visual testing tools take a screenshot of the UI in the testing process and compare it with a previously stated reference template. These test automation tools mostly help check for visual bugs, such as the wrong size, layout, or color of visual elements, overlapping visual components, incorrect/missing images and icons, but can also be used to check functionality. Screenshot-based automation can be easier and faster than pure scripting.
Applitools is leading the way in visual testing automation. In addition to powerful AI-driven testing capabilities that allow ignoring non-critical inconsistencies in UI visuals, Applitools offers parallel test execution, both on-premises and cloud-based runs and concise testing reports. It integrates with popular CI tools (Jenkins, Travis CI, CircleCI, and others), Jira (issues can be created and managed directly from the Applitools test manager) and Azure DevOps Server. However, Applitools is free for open-source projects only.
Other options: Screenster, Eggplant AI, Percy, Happo.
Snapshot testing tools
These test automation tools help exclude unintended changes in the user interface much faster and report them to the developers easier than with screenshot testing. They capture a complete UI structure of an element stored within text files and compare it with a pre-stated standard using a diff algorithm.
Other options: Ranorex.
They help verify that UI components (fields, titles, popups, texts and buttons) are correct. Some test automation frameworks (Jasmine, Jest) have them in-built, and in other cases, you need a stand-alone tool.
Other options: AssertJ, Unexpected.
The dedicated tools help check directly how the requests to an application’s business logic or external web services are processed and addressed. This allows for checking integrations and an application’s functionality 'fast and early in development.
- SoapUI / SoapUI Pro
Postman is a popular API testing tool that supports running scripts from the command line. It has advanced features, such as API data visualization, API mocking, built-in support of variables, API monitoring, and convenient workspaces to share and organize work between team members or multiple teams.
- REST Assured
REST Assured is an API to simplify REST services testing in Java. REST Assured is much appreciated for its flexibility and easy maintainability.
Also: JMeter, Fiddler.
Tools for this testing type ensure low-level code correctness and test a single unit of code.
- JUnit 5
JUnit is a leader in unit testing for the Java programming language. JUnit belongs to the family of xUnit frameworks, the testing model of which is on the way to becoming a standard framework for any language (PHPUnit, CUnit, NUnit, CPPUnit, PyUnit, etc.). JUnit is supported by all major IDEs and build systems, such as Ant, Maven and Gradle.
TestNG is a member of the xUnit family for Java inspired by JUnit, but it offers a more powerful functionality. Among the tool’s gains are more advanced and easier-to-use annotations and support of parallel testing.
Other options: MSUnit, NUnit.
Mobile testing tools help test native and hybrid mobile apps since not all web UI testing tools support testing of applications designed and developed for mobile devices.
Appium is a widely popular open-source and cross-platform tool that works for both Android and iOS platforms. Appium allows running automated tests on both physical devices, emulators and simulators. Based on Selenium WebDriver, it supports any programming language. The specific feature of Appium is that it allows invoking another application from the application under test, such as a camera app, for example.
Calabash is a mobile test automation tool that allows testing Android and iOS native apps and hybrid applications. Calabash has an in-built function for taking screenshots in a determined instant.
Espresso is an open-source testing tool by Google created to test user interfaces of native Android apps. Espresso tests both real devices and emulators. The main advantage of Espresso is faster test runs than with other mobile test automation tools and improved test coverage (up to 99% of bugs can be revealed).
Tools for running tests in the cloud help conduct testing for several browser/OS/device combinations without additional costs and efforts for the large test lab setup and support and for testing in parallel (especially important for massive regression testing).
- Sauce Labs
The Sauce Labs platform provides a ready infrastructure to run over 700+ browser and operating system configurations, including those for mobile and tablet devices (simulated browsers and mobile devices, emulators and real mobile devices). In addition, the platform has a screen recorder, which captures and stores the emulated actions in the image and video formats, and in-built test analytics.
BrowserStack is a cloud-based test automation platform that offers a large pool of browsers, OSs, and devices. It can also generate screenshots and videos of emulated actions. BrowserStack is easy to configure and use, and rather cost-effective. However, it does not allow re-using code written for Selenium in Appium-based testing.
Other options: Perfecto, LambdaTest.
Reporting tools help reduce the defect life cycle, simplify defect investigation and track down failures faster, as well as allow for the involvement of all testing stakeholders.
BDD testing tools help use the natural language to describe the behavior of the features under test.
Basically, SpecFlow is a Cucumber for .NET. SpecFlow is an open-source tool that reads text specifications in Gherkin and maps them to C# code to verify those specifications.
Other options: Cucumber-JVM, JBehave, JGiven.
You may use a test automation framework without in-built test runners or employ a third-party automated test runner to leverage capabilities that are more advanced.
Ava is a minimalistic test runner used for Node.js projects. Ava’s advanced capabilities include parallel test runs through separate processes, ‘fail fast’, storing snapshots (supports Jest snapshots), skipping tests and showing test coverage reports.
Other options: Wallaby, InternJS.
Codeless scripting tools capture the way a tester interacts with software and replay it on demand.
mabl is a paid ML-driven cloud-based tool for automated testing. It tests web apps, APIs, email and PDFs. mabl goes with test self-healing – tests adapt to UI changes automatically. The tool supports scheduled test runs and offers comprehensive test results reporting with screenshots and detailed diagnostic information. mabl tests can be created only in Chrome. However, the createdtests can be re-used across other browsers (Firefox, Safari, Internet Explorer).
Testim is a commercial AI-driven record-and-replay tool platform. Testim supports dynamic locators and can run tests both on-premises and in the cloud, but it offers only high-level reporting.
Test management tools help plan and structure the testing process and its phases, coordinate manual and automated software testing efforts, store information on testing activities and progress and report the state of quality assurance.
Zephyr is the most popular test management tool natively embedded in Jira, which means you can have test issues created, tracked and reported just like other Jira tasks. Zephyr provides both forward and backward traceability reports, allows for linking requirements and test cases, can run a group of tests for a particular project version (to test a cycle/sprint deliverables), offers execution reports and provides a test board with multiple individual configurable quality metrics. However, Zephyr does not have a feature to generate test suite/requirement-based reports.
PractiTest is a cloud-based tool for end-to-end test management. PractiTest is easy to use due to intuitive GUI and hierarchical filters to organize information. It is packed with features, such as an ability to report issues directly from emails, external dashboards and embedding graphs support, and session-based test management. PractiTest integrates with Jira, Bugzilla, Mantis; Jenkins, Slack, Zapier and more.
Other options: TestRail.
These tools help quickly get realistic test data like names, phone numbers, IPs, emails, addresses, filenames, commerce, company names, finance, images, and more.
Mockaroo is an online service for automated data generation. Mockaroo can provide test data in CSV, JSON and SQL formats. It generates up to 1,000 rows of data of various types in the free edition and up to 10 million rows in the paid version.
By aligning the development and testing environmnets, these tools help regularly run tests after every change in software (either every time changes are submitted or at a scheduled time, say, during night builds) and automatically prepare the code release.
Jenkins is the leading open-source CI server written in Java, which supports both on-demand and scheduled tests. Jenkins is better suited for enterprise-scale projects, as it offers the needed flexibility. At the same time, it requires time to set up and configure, and the dedicated staff to run, support and customize the environment.
- Travis CI
Travis CI is easy to set up thanks to pre-installed services and easy to maintain thanks to clear and detailed documentation. Travis CI comes with cloud-based hosting that allows testing against various environments, various machines and OSs. Travis CI supports cluster and parallel testing. The tool is free for open-source projects only.
Other options: Bamboo, CircleCI, Bitrise CI (mobile).
Stay on Top of Test Automation
The popularity of test automation goes hand in hand with the emergence of new tools aimed to make test generation, execution, and maintenance even more efficient. Obviously, our list cannot fit all existing test automation tools, but we will try to keep it updated with the most trending tools.
Every project has its specifics in terms of functionality and target users. We offer software testing services tailored to your business needs.