Critiquing Color-Coded Approaches in Information Security

How many “yellows” is a “red” worth? I know that is not an answerable question, but yet it is something that infosec regularly has to wrestle with. Let’s say we have two vulnerabilities, a yellow and the hair-raising red. When we call one vulnerability “yellow” and another “red,” the implicit claim is that “red” has more weight. But how much more? Is one “red” worth three “yellows”? Ten? The exercise collapses the moment you ask it seriously, because colors are not units of measure. They’re decorations. And yet, this is the scaffolding upon which most infosec vulnerability management is built.

‍

How about “A B C D E F” or “1 2 3 4 5 6 7 8 9 10” or “Critical, High, Medium, Low” or any other way your vendor wants to create a dashboard? It’s all the same, just different representations of what I refer to as stoplight infosec. 

‍

Stoplight infosec is the act of forcing uncertainty into artificial bins such as “Critical, High, Medium, Low” then pretending we can do arithmetic with those bins. All you can do is rank-order them with such a system.  You need more context if you want to do anything more, and even rank-ordering them is incredibly fraught with issues.

‍

Some vendors average stoplight infosec, thinking that somehow makes it better. If you have 100 “A”s on 100 different assets and one “F” on another, what is your score? Through averaging, you get an “A”! Congratulations. Of course you’re probably going to get hacked with that “F”, assuming its grade is representative of real risk of loss, but congrats on your “A”, I guess. But what if you have two assets, one with an “A” and the other with an “F”. Well your average is a “C”. Not so great. So what do you do? You have the perverse incentive to add assets! Just add a bunch more attack surface that has nothing on it, and you can mix that “F” in with the newly minted assets that are all “As” and you get an average score of an “A”.

‍

Congratulations for:

• increasing your attack surface 
• not fixing the “F” 
• and still getting an “A”

‍

See how ridiculous this is?

‍

You may be saying, “But Robert, what if it is a number, like a criticality of 10? That’s math. It’s a real number.” So, your overall risk is calculated to be ‘10 x 1 = 10’. Are you telling me the one world-ending vulnerability ranked at a 10 that you have determined will sink your billion dollar company is worth the same as if you fixed 10 vulnerabilities ranked at a one that account for literally no potential for loss? Because that’s what that math says. Which is again why this isn’t actually math, or not useful math anyway.

‍

CVSS is one such example. If I fix a vulnerability with a CVSS score of 10, have I done the equivalent of fixing five vulnerabilities ranked CVSS two? Or two vulnerabilities ranked CVSS five? None of that makes sense. It is because it is trying to predict what adversaries are going to do, and it turns out adversaries don’t care about the way we rank things. This is stoplight infosec in a nutshell. It’s not math, it’s people trying to attach math to non-math things.

‍

The true risk officer of the company is the CFO, and they have to deal with the aftermath of our inability to talk in actual business terms. If you walk into a CFO’s office and say, “I want $50k to fix 3 cross site request forgeries, 2 cross site scripting exploits, 1 SQL injection, and 1 command injection so that we change our grade from a B- to a B+” they will smile, nod, and show you the door. None of that makes any sense. It also, incidentally, is the kind of language that we used to use in grammar school. B-? I think it’s time for our industry to grow up. 

‍

The CFO would like that too.

‍

Alternatively, we could have a much more professional conversation with the same CFO, like this: “I want $50k to retire $9.4M in risk.” To which the CFO is going to want to see your math. Because the business speaks in the context of dollars and cents - because those are reducible and accurately represented in math. But the cool thing about math is that it is made of testable variables, and you can show your work, argue about details like what the actual fully loaded headcount cost is with the CFO and they’ll improve your math.

‍

Now to deal with the nay sayers. People will say, “But Robert, you can’t precisely know what the expected cost of remediation is, or the cost of the damage, or the true likelihood of exploitation.” To them I say, you are right, and you are very wrong.

‍

What is a car worth? If you were to look at a car you could reasonably expect to get close to the correct valuation of a vehicle within a few minutes of looking online. Now, you will be wrong, because the seller will negotiate to a different price point perhaps, or because it has a newly-identified feature that makes it unique and rare, or because it is wanted in a high profile murder or whatever random thing drives the price up or down. So while I can’t know for certain what the price of anything is, somehow market forces allow us to transact, which means prices can be identified, averaged and known to a fairly high degree of certainty.

‍

If you were able to read Jeremiah Grossman’s “We Are Security Optimists” post, you know that we see a green field of opportunity for our industry to adjust into the business context of dollars and cents - and sense. Throwing your hands up and decrying that we can’t know the value of things, or the risk of things, or the cost to do things is giving up before we even start. It is the old way of thinking, and it’s at an evolutionary end.

‍

Technographic information such as what things are installed could make knowing the cost of remediation rather straightforward. For instance: can they roll out a rule to a WAF to block that exploit, or an ACL that is accidentally misconfigured that could be fixed to remove hundreds of machines that were accidentally exposed to the Internet? These are incredibly simple examples but they give you some idea of what I mean when I say that we haven’t barely scratched the surface of knowing the true cost of things, let alone starting to measure the work of people who roll out those fixes to see what the average time it takes to perform tasks are, and on and on. To put it simply, we’ve not even tried this approach and yet it has so much potential.

‍

As a challenge, I think if the security community really put its mind to this problem it could make great headway to finding improvements to more accurately measure the cost of things and enable an entirely new way of cost-benefit analysis. We could more accurately decide which risks we can live with and which we can’t. We could also improve the value of companies by reducing the potential loss that will occur. 

‍

But as Security Optimists, we believe such a future is on the near horizon if people start thinking about this new paradigm and leave stoplight infosec in the past where it belongs. Stoplight infosec is a relic, a crutch for people who aren’t ready to move to where we have to go as an industry. The future, if we choose to build it, is one in which security teams communicate in the only language capital respects: risk denominated in economic terms. That’s not pessimism. That’s the optimistic view. 

‍

Because it means there’s a vast, green field of opportunity waiting for those willing to leave the colored dashboards and school yard grades behind.

‍