EMA (Enterprise Management Associates) recently released a report titled “Disrupting the Economics of Software Testing Through AI”. In this report, author Torsten Volk, Research Director at EMA, explains why traditional approaches to software quality cannot evolve to meet the needs of modern software delivery. It highlights five key categories of AI and six test automation critical points that AI addresses.
We sat down with Torsten and talked about the report and his insights into the impact of AI on software testing:
Q: What’s wrong with the current state of testing? Why do we need AI?
Organizations that depend on traditional testing tools and techniques fail to keep up with the needs of today’s digital demands and quickly fall behind their competitors. Growing application complexity and time-to-market demands across the enterprise make it difficult for software delivery teams to keep pace. There is a growing need to optimize the process with AI to help eradicate mundane and repetitive tasks and control quality costs that have spiraled out of control.
Q: How can AI help and with what?
There are five key features where AI can help: creating smart scribbling/NLP (Natural Language Process) based tests, self-healing, coverage detection, anomaly detection, and visual inspection . The report I wrote highlights six critical points where these capabilities can help. For example: false positives, test maintenance, inefficient feedback loops, increasing application complexity, device proliferation, and toolchain complexity.
Leading organizations have already embraced some level of AI-based self-healing and test creation, but by far the most impactful is visual inspection (or visual AI), which provides comprehensive and accurate coverage of user experience. It is able to learn and adapt to new situations without the need to write and maintain code-based rules.
Q: Are people embracing AI?
Yes, AI adoption is on the rise for many reasons, but for me, it’s not that people aren’t adopting AI – they’re embracing AI-based technical capabilities. For example, people want the ability to automate NLP-based testing for a specific use case. People are more interested in the return on investment achieved through the speed and scalability of using AI in the development process, and not necessarily how the sausage is made.
Q: How is the role of the developer/tester changing with the implementation of AI?
When looking at test automation, developers and testers need to make a decision about what belongs in test automation. How is it classified, for example. Then all you have to do is set the framework for the AI to operate and provide it with feedback to continuously improve its performance over time.
Once that happens, developers and testers are freed up to do more creative, interesting, and valuable work by eliminating the toil of mundane or repetitive work — work that has no value on its own but needs to be done. correctly every time.
For example, examining thousands of web page renderings. Some of them have small differences, but they don’t matter. If I can get the machine to filter out all the unimportant ones and just highlight the few that may or may not be a defect, I’ve now reduced my work from thousands to one very small handle.
Automatic classification is a great example of being able to reduce your work. If you reduce repetitive work, it means you don’t miss anything. Whereas if I’m looking at the same thing, which looks like the same page every time, I might miss something. Whereas if I can ask the AI to tell me that this page is slightly different from others you’ve viewed, and here’s why, it eliminates repetitive and mundane tasks and reduces the chances of error-prone results.
Q: Should I hire AI experts or develop an internal AI practice?
The short answer is no. There are many vendor solutions available that allow you to leverage the AI, machine learning, and training data already in place.
If you want to implement AI yourself, you actually need people with two sets of knowledge in the field: first, the field you want the application of AI to be, but second, an understanding deep dive into the possibilities of AI and how you can chain them together. capabilities together. Often it is too expensive and too rare.
If your primary deliverable is not the AI deliverable but the ROI deliverable that AI can deliver, then it’s best to find a tool or service that can do that for you and allow you to focus on your area of expertise. This will make life much easier because there will be many more people in a company who understand this area and only a small handful of people who only understand AI.
Q: You say visual inspection capability is the biggest impact – how does that help?
Training deep learning models to inspect an application through the eyes of the end user is key to removing much of the mundane repetitive tasks that make humans inefficient.
Intelligent exploration, self-healing, anomaly detection, and coverage detection are each point solutions that help organizations reduce their risk of blind spots while reducing human workload. But visual inspection goes a step further by aiming to understand application workflows and business requirements.
Q: Where do I start today? Can I integrate AI into my existing test automation practice?
Yes – Visual AI Applitools example.
Q: What is the future state?
Autonomous testing is the vision of the future, but we have to ask ourselves why don’t we have a self-driving car yet? It’s because today, we still chain models and models of models. But at the end of the day, where we’re striving to achieve is that AI takes over all tactical and repetitive decisions and humans think more strategically at the end of the process where they are more valuable from a commercial point of view.
Thanks to Torsten for spending time with us and if you are interested in reading the full report http://applitools.info/sdtimes .