Would it be a sacrilege to decrease the number of code tests? Surprisingly, there is a lot of data showing the benefits of less testing. A typical software test suite has the following drawbacks:
- The execution is too long
- Test code that has not changed
- Works in single thread
- Costs businesses countless hours and dollars
Software tests prove that the software operates to functional and non-functional specifications. It also provides feedback to developers on the quality of their code changes.
It is considered good practice to keep code changes small, thereby reducing the scope and extent of the change. Most continuous integration automations, however, launch full test suites that test thousands or millions of lines of code, even when a few lines of code have changed.
Automated test suites can take 12 hours to run and therefore may only run once per day. For others, automated tests can take one to two hours and run several times a day. In either case, developers have to wait for test feedback, even for simple bug fixes and cosmetic updates.
Tip 1: Your test suite should complete in ten minutes
Ideally, software developers write and deliver code 40 hours per week, but normal meetings and interruptions often push actual development below 30 hours per week.
One cause: developers spend too much time waiting. Once they have written the code, they wait for it to be revised; they wait for the code to be deployed; then they wait for the results of functional and quality assurance tests. Finally, they await validation, and integration tests.
It takes too long to put the code into production. Sometimes developers wait a day or more between each step of the coding process. Even when they’re “only” an hour or two away from receiving feedback on code changes, productivity drops dramatically.
In the meantime, developers can be tasked with low-effort bug fixes. There is a cost to this âcontext switch,â which interrupts the intense and creative focus required for analytical problem solving. Getting into this state of mind takes time and energy. For many, the change of context sabotages this productive mental state and decreases motivation.
Whether it’s twiddling their thumbs or switching contexts for an hour, developers get paid very well for waiting or creating less value. The solution: Dramatically speed up feedback on their work.
We observe that for many customers, 10 minutes is a minimum acceptable waiting threshold. It’s short enough to grab a cup of coffee, stretch, take an organic break, or check in with a coworker, knowing that they’ll have the test results when this conversation is over.
Tip 2: Reducing Test Suite Run Time by 90% is Doable
Many are surprised to learn that it is entirely possible to reduce test suite run times by up to 90%. Take the example of one of our key clients. Their test suites took between 90 and 200 minutes until recently. Now they usually take 10 or 11 minutes.
How can this be? Why Aren’t All Test Suites Faster?
In the early 2000s, most web applications were written as “monoliths”. New functions have been incorporated into a constantly evolving code base. Yes, there may have been classes and objects and other performance optimizations, but most Fortune 500 companies on the web were building “macro services.” These monolithic code bases have grown to millions of lines of code with little abstraction between the main elements of the application.
While the discipline of application architecture has shifted to cloud computing, microservices, and containerization paradigms, testing has not kept pace. Testing is one of the most underrated (or ignored) parts of the software development lifecycle. Most test suites are built like the monoliths of yesteryear, unnecessarily, as test technology has evolved.
Test suites can be multithreaded, but often they are not. They can selectively run only the tests relevant to the code changes, but often all of the rest is run arbitrarily. They can be designed for intelligence, acceleration, and reporting, but rarely are.
A modern software approach to testing can lead to modern test results. It won’t take years of refactoring and management; in fact, it is not very complicated.
Tip 3: Acceleration of enterprise level testing can be implemented in a single sprint
The power of big business can be difficult to harness. It is no small feat to bring together several departments with disparate priorities, some of them directly in conflict. Despite high levels of goodwill and a determination to succeed, projects inevitably take longer and cost more than initially anticipated.
Why is this happening? Because large implementations require behavioral changes, infrastructure builds, tool updates, etc.
Development teams should take advantage of modern tools that do not impose any changes to their workflows and allow them to use the same commands, validation processes, and testing procedures. What changes is how long they wait for a test suite to complete. Ideally, preparing for a test should only involve adding a single gem or package to a dependency list.
Tip 4: Faster Testing Keeps Developers Productive, Happy, and On Board
Developer roles are bigger and more in demand than ever. A December 2020 jobs report from CompTIA notes that developer jobs accounted for 62,900 (16%) of the 391,000 new tech roles entering the market. Tech companies live and die off the quality and retention of their engineering staff and right now there is more demand than supply.
When developers can’t actually develop, they lose motivation and enthusiasm for their work. The wait for feedback and the busy work produce frustration. That’s when they start looking for another job. Conversely, the more time developers spend producing code, the more they like it and the longer you keep it.
It’s much easier to retain developers who have the tools they need to be productive, enjoy their work, and feel motivated.
Tip 5: Speeding up testing can drive the next leap in developer productivity
Over the past 20 years, developer productivity has been driven by microservice architectures, containerization, cloud infrastructure, and serverless. Our ability to create and run faster, more ephemeral code has progressed in the blink of an eye. Our ability to test has not worked.
Now it is possible to change that and in many organizations save thousands of dollars every week. Now that we’ve modernized the infrastructure, coding paradigms, and scaling, it’s time to overcome the hurdles developers face in creating code.
Accelerating testing can lead to major advances in developer productivity. Companies should examine the status quo in their development processes; should test suites take hours or minutes? Should developers switch context or stay focused on coding?
For those who think test suites should only take a few minutes, developers shouldn’t have to switch contexts, and code delivery times should be faster, we can show how this is achievable.
About the Author:
Yves Junqueira, CEO and co-founder, YourBase