NIST Testing Guide Targets Common Source of Software Bugs – GCN
[ad_1]
New testing help for the common cause of software bugs
NIST publishes tutorial on automated testing of multiple variables
The National Institute of Standards and Technology has developed algorithms for automated testing of multiple variables in software that can cause security vulnerabilities, and posted a tutorial on how to use the tools.
The incorrect or unexpected interaction of two or more parameters in software, such as inputs or configuration parameters, is a major cause of security bugs. But testing for these problems has been limited by the cost and complexity of testing the large number of possible combinations. NIST reported in 2003 that such problems cost the US economy more than $ 59 billion a year despite the fact that more than half of most software development budgets were spent on testing.
Research has shown that in many cases, the vast majority of these defects, from 89 to 100 percent, are caused by combinations of up to four variables, and virtually all are caused by no more than six, NIST reported. .
Associated coverage:
NIST test puts software analysis tools to the test
“This finding has important implications for testing, as it suggests that combinations of parameter tests can provide very effective flaw detection,” NIST said in the tutorial,“Practical Combinatorial Tests, Special Publication 800-142. ”
The variable pair test, while convenient, can miss 10 to 40 percent of system bugs, NIST said. But a lack of good algorithms to test a larger number of variables at once has made such testing extremely expensive and is only used for software with high assurance for mission critical applications.
The Automated Combinatorial Testing for Software program is a cooperative effort of NIST, Air Force, University of Texas at Arlington, George Mason University, Utah State University, United ‘University of Maryland and North Carolina State University to produce methods and tools for generating tests. for any number of combinations of variables. SP 800-142 provides instructions for their use.
New algorithms and tools make automated testing convenient for relatively small combinations of variables, but combinatorial testing is not free. The NIST publication provides information on the costs and practical considerations for each type of test, and explains the trade-offs and limitations.
About the Author
William Jackson is a Maryland-based freelance writer.
[ad_2]
