Measuring Cybersecurity: The Imperfect but Valuable Tools

As companies strive to enhance their cybersecurity posture, they are increasingly relying on a variety of metrics, scoring systems, and reputational rankings to assess their efforts. However, in many cases, businesses are placing excessive demands on the systems designed to measure security. The old adage states that you must measure something to manage it, but many systems, such as the Common Vulnerability Scoring System (CVSS) and organizational security posture scoring, are often only effective at expressing measurable risk.

The Imperfect but Valuable Tools

Corporate boards are now using security measurements as key performance indicators (KPIs), and some industries, like insurance firms, are utilizing them to determine risk. The conclusion is that scoring risk and reputation tools are imperfect but better than nothing. This is partly because companies aim to manage risk, not just improve security, according to Bruce Schneier, chief technology officer of Inrupt and an adjunct lecturer at the Harvard Kennedy School.

Comparative Metrics and Risk Management

Schneier emphasizes the importance of building comparative metrics to gauge how a company compares to its peers. "Whenever I’ve had a company that could do it, I’ve always tried to build comparative metrics — how am I doing compared to everybody else that does this?" he says. This approach helps people understand how they compare to their peers and provides valuable lawsuit protection.

Growing Efforts to Assign Scores and Reputations

Efforts to assign scores and reputations to various components of the information technology ecosystem are expanding. Detection and response platform Sweet Security has partnered with OpenText to utilize the Open Source Select service, which uses ratings for contributors, popularity, and security to summarize open source component practices. For example, TensorFlow received green ratings for its contributors (score: 73) and popularity (score: 84) but only a yellow rating (score: 42) for security.

The Benefits of Machine Learning

The biggest challenge in using these metrics effectively is understanding how to use the vast amounts of data in an effective way. According to Thomas, senior director of product and engineering at OpenText, machine learning algorithms can provide significant value in augmenting security decision-making and synthesizing data into meaningful patterns.

The Importance of Measuring Effectiveness

The question that companies should ask when using metrics is whether those metrics are speeding up decision-making processes. As Thomas notes, "The benefit is, as a developer, I’m not waiting weeks to work through an open source intake process. I can quickly get a decision in a subset — and, hopefully, a meaningful subset — of use cases." The key is to ensure that metrics are not just measured for the sake of measuring but also to measure the effectiveness of those metrics.