*2.3. Security Metrics*

Standards of measurement and metrics are a powerful tool to manage security and for making decisions [70–72]. If carefully designed and chosen, metrics can provide a quantitative, repeatable, and reproducible value. This value is selected to be related to the property of interest of the systems under test (e.g., number and distribution of vulnerabilities). The use of metrics enables results to be compared over time, and among different devices. In addition, they can also be used to systematically improve the security level of a system or to predict this security level at a future point in time.

Although the capabilities of metrics have been demonstrated, they are not free of drawbacks. In our previous research work [72], we performed a systematic review of the literature and standards. To detect possible gaps, our objective was to find which types of metrics have been proposed and in which fields have been applied. This research work concludes that, in general, standards encourage the use of metrics, but they do not usually propose any specific set of metrics. If metrics are proposed, then they are conceived to be applied at a higher level (i.e., organization level), and then cannot be applied to industrial components. This type of metric is usually related to measuring the return on security investment, security budget allocation, and reviewing security-related documentation.

Our previous results also highlight that scientific papers have focused their efforts on software-related metrics: 77.5% of the analyzed metrics were exclusively applicable to software (e.g., lines of code, number of functions, and so on), whereas only 0.6% were related exclusively to hardware (e.g., side-channel vulnerability factor metric). In addition, 14.8% of them could be applied to both software and hardware (e.g., the historically exploited vulnerability metric that measures the number of vulnerabilities exploited in the past), and the remaining 7.1% are focused on other aspects, such as user usability. This shows that there is a clear lack of hardware security metrics in the literature, and the main contributions are centered on software security.

Other research works also reveal common problems across security metrics [73,74]:


Under this scenario, it seems reasonable that future research should be focused on the development of a convincing theoretical foundation, empirical evaluation, and systematic improvement of existing approaches, in an attempt to solve the lack of widely accepted solutions. In this research work, metrics constitute a key element. They are developed to analyze the distribution of vulnerabilities and to track their evolution over time.
