Test the software, debug the software, test the software, debug the software … repeat as often as needed, until the software is clean and bug-free – or at least as close to bug-free as possible .
Software testing is a chore – an essential chore, but a chore nonetheless. At least one company has seen a gap in the market that could clean up the software before testing has even started. The company is Tricentis and CEO Sandeep Johri spoke to ZDNet to explain how it’s done.
ZDNet: You are highly rated in software tests by Gartner and other analysts, what’s your secret?
Johri: Software testing is nothing new. Whenever you develop software, you should test it before you release it. But most of the time, believe it or not, testing is done manually in large companies. About 50 percent is done overseas, but most are done manually.
SEE: How to Build a Successful Career as a DevOps Engineer (Free PDF)
We provide a continuous testing platform that makes you highly automated, and we can automate pretty much any software. This gets you to the point where your tests run at the same rate as your progress cycle, so you can run daily tests and testing doesn’t become a bottleneck.
We have what we believe is a single platform that will offer a 10x improvement over a traditional manual testing solution.
We not only increase the speed by reducing the testing time, but also, because it is highly automated, it is not too expensive. So you can reduce the cost of testing, which is typically around 40% of the cost of an application platform.
Analysts estimate that companies spend more than $ 30 billion on testing, and most of it shipped to India or the Philippines or a similar location, where they install the software on computers and perform the testing.
The main differentiator – the reason we can do this and some not – is that most commercial tools are script-based tools, and while you can automate anything you want with scripting, scripts can be very fragile and break all the time. This means that if you go through a rapid development cycle, your scripts will crash so often that you won’t know if it’s the script or the software that is not working.
So you drop that testing, drop that automation, and go back to manual testing. We’re a scriptless solution – that’s at the heart of what we do – and because of that, our solution is resilient.
There are other capabilities you need to achieve this level of automation. It’s about managing test data – making sure every scenario is covered – and that means you’ll end up having to prepare a lot of data. Think about the banks that have to test every scenario, every circumstance for every type of customer. Test data is becoming a big deal, but we have the ability to make it much more efficient.
For example, when you try to test an app, not all of the items in that app are always available. Say you are an insurance company and you want to redo the processing of your claims. This business process test is based on almost all applications of this insurance company. So how do you test this? In this complex environment, we can virtualize the applications, so you can still test your small sub-application without having all the other applications available.
You personally have a very distinguished career, I notice that. You served as co-chair of President Bill Clinton’s National Information Infrastructure Advisory Council. What drew you to the testing area?
The main reason was that I spent about eight years at HP – 2003-2011 – and held various positions in IT management where I was in charge, and I was also responsible for acquisitions. While I was there we made 14 acquisitions, valued at almost $ 7 billion, but the biggest acquisition we made was a company called Mercury Interactive. It was one of the best acquisitions HP has ever made – and that earlier battery life, which you know turned out to be a complete disaster.
Mercury was a $ 4.5 billion company when we bought it and it was dominant in its field. Every business in the world used its software.
I ended up leading the acquisition of Mercury; I got to know the company and in 2010/11 HP stopped investing in Mercury. They had lost interest and I felt that Mercury, even though it was still dominant, was not following the more recent trend, which was mostly about automation.
Customers were embracing Agile and testing became a bottleneck, and my thesis was that there was an opportunity for a next-gen testing platform. This kind of opportunity often presents itself in the field of technology.
Agile development and automatic testing have become increasingly important with the advent of DevOps. So that’s what brought me here: I was in the business and I knew all the sales people. And I was aware of what customers were looking for that Mercury was not delivering.
I notice that one of the areas you are interested in is the Jenkins Automation Server. Why care about what many would see as a relatively obscure choice?
Jenkins is really interesting. It is an open source company and it is the open source tool of choice for doing builds and orchestrating their release. In fact, we fit in with Jenkins; what ends up happening is you have a bunch of people sitting down developing stuff, and once they submit all the code it’s taken, then you do a new build and go to the next cycle. .
We integrate with them and we have clients who integrate with them; what customers do is grab the code, which triggers our application, Tosca, to run automated tests. Now you can ask Jenkins to basically say, “a new version is ready, so let’s get it going.”
We now have clients like WorldPay which is a UK company which is now a global company and is the largest credit card processor. Their US branch was one of our clients, and what they did was go from six to eight week release cycles for all platforms to one every two weeks.
They develop on Agile but now, every day at the end of the day, when the developers have done all their code, Jenkins picks it up and runs the automated tests. And it happens every day.
What happens is that every day you make a new version that, even if you don’t release it, you run a bunch of automated tests, and in doing so, that means by the time you get to by the release date, you’ve dealt with most of the bugs.
Otherwise, what used to happen is that every day the developers did their little tests and everything was fine; then, after two weeks, when they were about to go out for the first time, everything would fall into place and hell would break loose.
And this is where Jenkins is so good. Think of it as a release automation – or release orchestration – tool that, on the one hand, helps do the build and, on the other hand, for us, prompts us to run a series of tests.
What do you see happening as you go along? Are there other areas in which you will develop?
We are now recognized as a leader in space. We have over 1,000 customers, from the very small to the biggest companies – some of the biggest banks, some of the biggest insurance companies, the biggest telecommunications operators. We are growing very well and we hope to maintain our leadership position.
In addition, we have made acquisitions – three in the past 18 months – to complete the portfolio.
Like others, we are building a capacity for AI and machine learning, just to make it smarter. As you make testing more resilient, you can automatically test to see which tests aren’t needed, which tests are most important – there are some really interesting innovations going on around machine learning and machine learning. AI.
We have a whole set of capabilities that we have developed on this front.
Can you tell us about clients you want to highlight?
HSBC is a customer, as is Exxon. In the UK we have Centrica, British Gas; and in Australia, Telstra, as well as three of Australia’s four largest banks.
You briefly touched on DevOps: How do you think this whole area is changing? Do you think DevOps is the way to go?
What has happened there is that it is mainly used as a methodology in a business – what people call âEnterprise DevOpsâ. When people think of DevOps, they think of a Netflix making 20 releases a day or whatever they do.
When you think of large companies, it’s not about doing 20 releases a day, but the ideas are the same: how to get the releases out?
SEE: How to Build a Successful Developer Career (Free PDF)
So on the developer side, people use other methodologies like behavior-driven development, test-driven development, and then there are newer methodologies. But when they’ve done all of that, they’re still having issues on the testing side – and that’s where we come in.
We’re developing faster, but we don’t need to publish faster and the next bottleneck is that we need to fix this. Because with fully automated testing, you can’t really post any faster.
And people offsite are migrating to the cloud and setting up real-time monitoring, which means when you kick things into production, you get a better idea of ââhow things are being used.