Risk Assessment COMbined with Automated Testing

The Challenge: Risks of complex ICT systems
Networked information and communication systems (ICT) conquer our daily lives and change the industry. Critical infrastructures, such as power grids or the banking system, are nowadays already heavily dependent on them. The importance of ICT systems will continue to rise with autonomous means of transport (e.g. autonomous cars and trains, drones) in the near future. The direct, existential well-being of persons is thereby increasingly entrusted to information and communication systems. The requirements for their security and reliability are correspondingly high.
In order to capture and minimize the risks, security-critical ICT systems should be subjected to careful risk management in accordance with the established industry standard ISO 31000. However, for complex systems, risk management can be very complex and difficult. While the subjective assessment of experienced experts can be an acceptable method for risk analysis in small scale, other approaches need to be chosen as size and complexity increase. One possibility for a more objective analysis is the use of security testing according to ISO 29119. However, the testing itself can also be complicated and expensive, in particular if unintended, unknown behavior is to be analyzed. Even highly unsafe systems provide many harmless test results, as long as the “wrong” test cases are created and executed.
The idea: Combine risk assessment and security testing
One way to deal with these difficulties is to combine the different approaches and try to use the strengths. In a complex system, it is first necessary to carry out a high-level assessment of the risks based on experience and literature. In order to make this initial risk assessment more precise, it is possible to use security testing exactly where the first high-level risk image shows the greatest uncertainties. The objective test results can then be used to extend, refine or correct the previous risk image. Economically applicable to large complex ICT systems, however, this method is only provided with adequate tool support.