Every code change carries the risk of causing unwanted side effects that lead to error states in the software. In order to find such errors at an early stage (and not just in production), automated testing must validate both previous and new software functionality.
First, the project goals (vision) and an approximate roadmap should be defined. The necessary budget should not be neglected here. As expected, the effort involved in developing fully automated tests is much higher early in the project than later in operation. However, some “background noise” still remains, as test results need to be evaluated, test cases adapted and new test features implemented.
Once the procedures and processes of the software to be tested have been clearly defined, the tools and frameworks are selected. Subsequently, the first prototypes should be presented during a launch in order to have the tools used accepted.
Automated Software Testing – An Introduction
After successfully building the (new) software, the test automation is automatically started with the version of the software to be tested. In the case of installable software, this must be done without manual intervention in so-called “silent” mode. By the way, most software installers offer a “response file”. A response file is a text file that contains the dialog boxes and corresponding inputs that the user specified during the first installation.
The software is then automatically configured and parameterized, and the proper functioning of applications, services, the browser, the graphical interface, etc. is tested. If the software product to be tested supports different platforms (Unix, Linux, Windows), all these operating systems must be tested, at best in parallel and in nightly execution. In this way, long-running tests can be performed efficiently, hardware resource utilization is better distributed, and test results are available in the morning for further analysis.
If a large number of tests run in parallel on several different test systems, the use of a virtual computer system, or virtual machine (VM), is recommended. If you don’t want to use your own hardware, cloud computing is an attractive alternative. Here, the necessary computer resources are also provided.
Test with the Robot Framework
For the development of fully automated software tests, the Robotic frame is definitely my favorite. The decisive factors are the extensive modular system to extend the range of functions, as well as the simple handling with good integration possibilities. Another plus point is the simple, tabular structure of the test cases, which requires only a short training period.
The Robot Framework focuses on user testing (acceptance testing and end-to-end testing) but is also used for system testing and smoke testing. The Robot Framework, developed in Python, is freely available as open source software (Apache license). A new major version, Robot Framework 5.0, was released towards the end of March 2022.
Installing the framework is simple, as only Python 3 and its “pip” package manager are required.
Two established systems are available for development and test execution, but they can also be used together to take advantage of each environment. WALK is Robot’s own development environment, making it easy for newcomers to learn how to use the framework. RED is an Eclipse-based development environment (IDE) that will delight accomplished software developers.
Test automation—scenarios
As described above, all software functions should be tested automatically. The Robot Framework helps us here with a wide range of additional libraries and packages that can be easily integrated. An example is the very stable SSH librarywhich makes it possible to run all the tests remotely and reusable under Windows, Linux and Unix.
In the field of end-to-end user interface (UI) testing for web browsers, the new and modern browser library from Playwright (since early 2021) or the long-running Selenium frame are available. These libraries can be easily integrated into the Robot Framework. This makes it easier to test graphical functions such as mouse clicks or user input and output on websites.
Of course, self-written test scripts can also be seamlessly integrated into the Robot Framework. Regardless of whether Java, Python, or shell scripting is preferred, only return values of 0 or not should be observed for useful reporting.
Software testing—assessment
Complete test reports are automatically generated as HTML pages by the Robot Framework. Additionally, test results can be imported into a test management system such as Test Link. To be able to analyze errors more quickly, a precise description of the error with output of log files is very useful. If there are a lot of tests, it is also a good idea to define the responsibilities for the tests. This allows the error report to be sent to the corresponding colleagues via an automated e-mail.
The icing on the cake for the graphical presentation of test results is the integration with, for example, a Splunk Dashboard. This makes it possible to comfortably see the current situation, especially with a large number of software tests.
If the dashboard is not enough, a traffic light system is the best solution. With the previously defined threshold values, the different colors of a USB traffic light can be used. For example, if you arrive at the office in the morning and see the red light flashing, it’s immediately clear: errors have occurred. Otherwise, if you see a green light, your day may start less urgently because all the tests have been run and the software has run error-free.
Fully automated testing is an indispensable factor, especially in agile software development, to ensure error-free software at all times. Valuable resources can be saved with test automations that run overnight. The Robot Framework is a great tool that can be used to automate all software testing.
