Shostack + Friends Blog


Zen and the art of not quantifying risk

Many people want their threat modeling work to produce risk numbers, and in this post you'll learn why that's a mistake.

Many people want their threat modeling work to quantify risk. This is understandable, but it turns out to be counter-productive. That's because the work to address a risk by changing a control or a design is not dominated by the risk, but by the change. If you have a risk that someone will steal data because there's weak authentication on an API that reveals confidential information and is used by thousands of customers, the need is real, and there's a business challenge in fixing it.

Quantifying risk in the wrong way creates avoidable problems. These include getting security "spun up" about about issues that are hard to fix, trouble with auditors when things which are "high risk" aren't fixed, and conflict between security and development, or security and operations.

This isn't to say that we should ignore real problems because they're hard to fix. It's not to say that we should ignore security risk — as in the API example, that's a real problem that should be fixed. But the fix comes after dialog with others who are involved in the system. For this, a "T-shirt sizing" may be enough. We aim to get involved in overall improvement prioritization. I first heard of this approach when hearing about a security person (I think Window Snyder) sat in war room for Windows XP SP2. War room was where decisions were made about various bits that either were or were not going to ship in the service pack (or other release). Having representation there mattered.

There are two side effects of moving from risk management to integration with prioritization. The first, frankly, is more work for security. Someone has to make time for the meetings in which priorities are set. (That you/y'all have something better to do may say a lot, and it's not pleasant.) The second side effect is that you may see more lows fixed. There are issues which are easy to fix "while the hood is open." Sometimes, in the process of fixing one issue, you can cluster a set of fixes together.

Lastly, there's a variant on the usual small/medium/large, and that is possible, plausible, probable. (Along with "pagers-going-off" and "post-mortemed".)