Why fixing software bugs should be the CEO’s problem

CEOs and other leaders can and should be intimately involved in software quality, just as they are with business and finance divisions. This involves understanding the functioning of technical teams and setting up a quality management system. You can do this by asking two simple questions about a recently shipped product. First, ask what criteria were used to determine when the product was ready to ship. There should be a clear discussion of how long to test the system, what type of testing is done, and the specific criteria used in the decision to ship the product. Second, ask what the current state of the defect is after the first six months. It’s normal to see an increase in defects after the initial shipment, due to increased usage in real-world environments, but leaders need to probe how engineering teams prioritize bugs and categorize the most serious defects. With this information, executives can explore the key metric of open days for a fault, to ensure that the most serious faults are corrected quickly.

Software platforms permeate the fabric of our lives, but only 27% of Fortune 100 CEOs have degrees in engineering and science. Take a quarterly earnings conference call and you’ll hear plenty of talk about revenue, expenses, and geographic trends, but little (if anything) about the quality of the company’s software. The results are obvious: For almost all major disasters caused by software defects, the autopsy usually determines that the defect has been around for some time.

The problem isn’t that business leaders should and don’t have engineering training, but few outside of engineering silos know how to discuss critical software systems. As a result, software bugs usually stay under the CEO’s radar unless a cataclysmic event occurs.

The sudden acceleration problem in Toyota cars was a textbook case of what can happen without a proactive internal quality system. Between 2004 and 2010, several investigations were opened to find out whether Toyota vehicles accelerated correctly; various theories have blamed electronics, entrapment of the floor mat, and “sticky” pedals. Toyota maintains that software bugs are not responsible for errors. When a lawsuit against Toyota was settled three years later, two expert witnesses retained by the plaintiffs said that “the system was faulty and dangerous, riddled with bugs and loopholes in its safety which led to the root cause of the problem. ‘accident “. A Toyota programmer described the engine control application as “spaghetti-like”. The interminable nature and lack of resolution of this case are clear signs that an inadequate quality review system was in place. With a good system, any bug would have been visible to leaders month after month, early and often, prompting effective – and life-saving – action. Since the complaints, Toyota has become much more proactive in addressing concerns. Unrelated to the acceleration issues, in February 2010, Toyota proactively recalled 437,000 cars to repair the braking software. Another positive outcome was that in a 2014 DOJ regulation regarding floor mats and sticky pedals, Toyota announced that it had made fundamental changes to its corporate structure and internal safety controls, including a team rapid response to investigate potential defects.

Business leaders can and should be intimately involved in the quality of software, just as they are involved in business and finance divisions. This involves understanding the functioning of technical teams and setting up a quality management system.

What does the opening of the way look like

I worked in the mainframe division of IBM at a time when IBM and Microsoft had quality issues that were causing significant business difficulties and customer satisfaction issues. In the early 1990s, IBM was having serious quality issues in the field in addition to difficulty meeting deadlines for new products. CEO Louis Gerstner Jr. was frustrated that on the expiration date of some products he was told they would miss the launch date by a year. In a legendary memo, Gerstner proposed an amnesty program – 30 days to reset dates, and after that missing a date would be grounds for dismissal.

The task of creating new schedules has been given to Nicholas Donofrio, the senior vice president of technology and manufacturing. Donofrio’s motto was “Be frank [about your schedule and quality issues] and i will be coming [about getting you the needed resources]. “He did not have direct line management for the systems business (both hardware and software), but since he was the former head of the business unit, he had deep knowledge and deep personal relationships. Almost 80% of product dates have been reset and every product organization has end-to-end quality management systems in place. In 1999, IBM was the world leader in servers, with 23% market share .

Likewise, in the 90s, Microsoft’s Windows operating systems had a series of bugs that caused computers to crash frequently. The public was largely used to these failures – “blue screens of death” – but Microsoft was subsequently hit by other bugs: As Windows systems connected to the Internet, they suffered a lot of hacks, viruses and problems. embarrassing security. Bill Gates responded to the crisis with a memo calling for arms, which was sent to 50,000 employees and simultaneously published in Wired. It defines “Trusted Computing,” a broad set of initiatives to improve product safety and design, and describes the prerequisites for a comprehensive quality management system (QMS), including changes. added to software design and development processes, new error reporting capabilities, and new update features. In 2000, Microsoft’s revenues exceeded $ 20 billion, and the company reported that customers cited the newly released Windows 2000 server as the most reliable version to date.

These two examples show how leaders can remedy deteriorating engineering divisions by asking simple questions, setting standards, and seeing themselves as part of the process.

When software is critical, so are bugs

Creating an effective quality management system that includes both engineers and top executives should be approached as an evolutionary exercise, simply starting with a focus on high impact from the start. CEOs should ask two simple questions about a product that has recently shipped:

1. “What criteria were used to determine when the product was ready to ship? There should be a clear discussion of how long to test the system, what type of testing is done, and the specific criteria used in the decision to ship the product.

2. “What is the current state of the defect after the first six months? It’s normal to see an increase in defects after the initial shipment, due to increased usage in real environments, but leaders need to probe how engineering teams prioritize bugs and categorize the most serious defects. With this information, executives can explore the key metric of open days for a fault, to ensure that the most serious faults are corrected quickly.

The 2 × 2 grid below shows how to assess a quality management system based on the answers to these two questions. Along the y-axis, you measure how siled the information is, based on how quickly questions are answered. Along the x-axis, you measure the organization’s response to your questions, based on the team’s proactivity to bugs. A “Mature” QMS can be identified by the prompt and serious sharing of information on faults with key leaders; in a “troubled” system, responses come forward slowly and with great antagonism.

If you find that your business is in the Troubled quadrant, your first step should be to immediately create a proactive focus on the flaws in order to move on to the Learning quadrant. This is what Gerstner and Gates both did. Your goal should be to create a culture where honesty is rewarded and employees feel safe discussing methods and techniques for remedying software defects. In this environment, information should flow easily between the required silos, moving your system from the Learning quadrant to the Mature quadrant.

Top-level quality management

If you are creating an QMS from scratch, your first step is to decide how the organization will categorize and prioritize bugs. This should be done by the teams in contact with the customers in collaboration with the customers. Typically, teams will favor two types of bugs: those that cause a system crash and loss of service (these were rated highest at IBM) and those that are less severe but could be ubiquitous.

Then, as an organization, decide your target response time for each severity level. If the QMS is new, the initial focus should be on fixing the most serious bugs within hours or days. As you use your system, you can collect data on two key metrics, inbound bug rates and bugfighter productivity, and adjust your goals as needed. Finally, you need to create an evaluation system that involves yourself and other senior leaders. Reviews of open defects and the time required to resolve a defect should be carried out in varying degrees of detail at all levels of the organization.

Once the QMS is established, the CEO is unlikely to see many old bugs just because no one wants to give the same excuse two months in a row. The CEO could also go through all the high-severity and most prevalent bugs. The CEO might ask the simplest question: “How was the software released with this type of bug?” The product development team might respond with software engineering jargon of ‘breakouts’, and the CEO might then ask an almost rhetorical question: “Are we going to test these conditions next time before we launch the product?” “

This type of QMS will improve the customer experience. To understand how important this is, let’s consider a software flaw that I regularly encounter in my bank’s online control system: An expense code function frequently fails, forcing me to restart my computer. I’ve complained countless times over the last year, and all the bank says is, “We told IT, but we don’t know if they’re going to fix it. This is a sign that the bank lacks a rigorous QMS, which leaves its financial advisers powerless and annoys its clients, possibly forcing them to change banks. I wish I could tell them that they can make a turnaround, like IBM and Microsoft did.

Editor’s Note: We have further clarified the paragraph regarding investigation of Toyota vehicle acceleration issues. We have also clarified that the expert witnesses cited in this paragraph were retained by the complainants.