Updates to National Institute of Standards and Technology’s (NIST) Automated Combinatorial Testing for Software (ACTS) research toolkit could better assist developers of safety-critical applications find errors.
By augmenting an existing software toolkit, NIST researchers hope to strengthen the safety tests that software companies conduct on the programs that help control vehicles, operate power plants and manage other demanding technology.
NIST Framework Used to Secure Railcar Purchase
Govt., Private Sector Need to Unite on Cyber: Report
Safety, Security, Privacy in Interconnected World
DDoS Attacks, Fewer in Quantity, More Sophisticated
ACTS is designed to help developers ensure that their products are safe from simultaneous input combinations that could trigger a dangerous error. A software error can have serious consequences in the case of safety-critical applications.
ACTS can help make sure that there are no simultaneous input combinations that might inadvertently cause a dangerous error.
Researchers from NIST – in collaboration with the University of Texas at Arlington, Adobe, and SBA Research – have found a way to properly test even software that has thousands of input variables.
ACTS toolkit now includes an updated version of Combinatorial Coverage Measurement (CCM), a tool that should help improve safety as well as reduce software costs. The software industry often spends seven to 20 times as much money rendering safety-critical software reliable as it does on more conventional code.
NIST mathematician Raghu Kacker said CCM represents a substantial improvement to the ACTS toolkit since its last major addition in 2015.
“Before we revised CCM, it was difficult to test software that handled thousands of variables thoroughly,” Kacker said. “That limitation is a problem for complex modern software of the sort that is used in passenger airliners and nuclear power plants, because it’s not just highly configurable, it’s also life critical. People’s lives and health are depending on it.”
Because a single input variable can have a range of potential values and a program can have many such variables, it can be a practical impossibility to test every conceivable combination, so testers rely on mathematical strategy to eliminate large swaths of possibilities. By the mid-2000s, the NIST toolkit could check inputs in up to six-way combinations, eliminating many risks of error.
“Our tools caught on, but in the end, you still ask yourself how well you have done, how thorough your testing was,” said NIST computer scientist Richard Kuhn, who worked with Kacker on the project. “We updated CCM so it could answer those questions.”