Some Thoughts on Binary Risk Assessment
Ben Sapiro showed off his Binary Risk Assessment (BRA) at SecTor recently. While I didn’t see the presentation, I’ve taken some time and reviewed the slides and read through the documentation. I thought I’d quickly give my thoughts on this:
It’s awesome and it sucks.
IT’S AWESOME
That’s not damning with faint praise, rather, it’s acknowledging that it’s not really “risk” but is a useful tool if your goal is to be quick and dirty about vulnerability severity.
In other words, this is much better than CVSS, and should probably replace it immediately.
TILTING AFTER THE WRONG WINDMILLS
In fact, it’s a shame that Ben chose to compare this to OCTAVE, FAIR, SOMAP and others. Because if he positioned this as “stop screwing around with CVSS” and “not really risk but a vuln rating” I would be telling everyone how much I liked it in that role.
In addition, if he positioned it with the Accounting/Audit Industrial Complex as good tool in the toolbox to compete with^H^H^H^H^H^H augment their RCSA nonsense, I could probably welcome it there, as well (though not as an optimal solution).
The power of BRA is the fact that Ben chose to make things “binary”. I can see this simple approach working well because it doesn’t allow you granularity – none of this arguing over “Moderate” or “Moderate-High” – just yes/no.
Also, Ben’s done a really good job thinking through what creates risk. For the FAIR familiar there’s the concepts of TCap and Control/Resistance Strength. I like that.
IT SUCKS
Speaking of subjectivity, I believe that Ben uses the power of binary choice to suggest that BRA “highlights” subjectivity. Not to be a rude pedagogue, but it really doesn’t “highlight” subjectivity as much as it just doesn’t give you many choices as to where to “put” that subjective measurement. Everything about it is still subjective (but that’s OK), and to reduce (or as I would rather “address properly”) that subjectivity would take more complexity than I believe Ben wanted to build (again, that’s OK).
As such at the end of the day, Ben’s right, it’s never going to be a replacement for what he calls “complex” analysis methodologies. And because it doesn’t properly address subjectivity, BRA is not for formal risk or threat modeling. I could never use it in my current capacity, as BRA just leaves a few too many questions unanswered. I don’t have time for the arguing, I just want your SME estimate, throw that puppy into OpenPERT and be done with it.
Furthermore, it’s odd because even though BRA suggests that it is designed to to “not ask anyone to guess on event frequency in the absence of statistical data (whatever that is)” it seems Ben’s intellectual honesty still could not let him escape the need to highlight it. If you look at BRA the model, that occurrence thing there, yeah, that’s frequency. It’s just a “binary” frequency determination which means….
BRA only talks about what’s possible.
As a risk model, this is the point at which we reference the Tacoma Narrows suspension bridge that oscillated wildly in the wind. Constructed nicely and all, but a small fundamental flaw in design renders it crazy bad for its purpose.
Also, impact is difficult for me to buy into because it uses asset value. I hate to break it to you, but asset value mainly matters to threat motivation modeling. The accounting value of the asset is RARELY the same as the losses we actually realize.
CHOOSE IT OR CHUCK IT?
So it wouldn’t be a review without such criticism. This is one reason I hate reviewing things, because it is a critical process. So please note that the above isn’t said with malice, it’s just an examination of the model itself.
In fact, as a tool, I wouldn’t dismiss it just yet. If your security group isn’t formally into risk, is stuck doing too much with CVSS for too little return, I’d jump all over this. If you have bigger fish to fry than an enterprise risk assessment but have the regulatory duty to create a risk register, BRA might just be the thing. If you find yourself faced with an absurd RCSA from audit or something – I might whip out the sweet BRA iPad app and run a scenario or two through. If I actually wanted a risk analysis, however, I would go elsewhere.
@alex, while I don’t disagree with your points, having sat in the presentation I thought I would add a little context here that may be needed.
In his presentation, Ben did a good job of verbally describing the intentions of the binary risk assessment. One of the issues he was tackling dealt with having a risk conversation where their only viewpoints of risk were (risk = frequency x impact). One of the benefits of the binary approach was to highlight where the differences were between the security practitioner and the business. This would ultimately facilitate a more meaningful conversation of the differences, something that can be very helpful when ‘negotiating’ a risk ranking in an organization.
I personally loved the simplicity of the back of the napkin approach and took it as just that. Not something that in any way would replace a FAIR or OCTAVE.
I see it as a great conversation starter.
I had to revive this old post, but it’s worthwhile.
Binary Risk Assessment is a good approach to conduct very rapid triage assessments. The value of the model is to adapt it so that the calculation makes sense in your environment. Moving down the analysis methodology determines the risk path (see Work Card).
I’m not sure that the Tacoma Narrows Bridge incident is a good analogy since the fatal flaw in the calculation involved an inverse function. BRA uses 1s and 0s. No inverse calculations happening here. Plus it’s yet another ordinal scale-based risk assessment that is driven by the use of simple binary options (on/off, yes/no, stop wasting your time/goof off.).
Try it out. Conduct several risk assessments with it and compare it to risk assessments using Octave Allegro, ISO 31000, and/or FAIR and I think that you’ll be surprised to find the precision of the results between the methodologies. Best part, your risk assessment window went down from days, sometimes weeks, to minutes/hours and know that you have fidelity of results to determine risk.
Phil,
In the spirit of the New School, if you’ve done those comparative risk assessments, why don’t you publish them, and save everyone else from having to invest the time in replicating the results?
Well put, Alex. BRA is an interesting approach, and as such might be worth pulling out of the risk management toolbox when it suits a given situation. Personally, I’m comfortable jumping directly to the end-game, assessing the probability and impacts directly and discussing my reasoning on risks with anyone who cares to discuss it – but then like you, and Ben, and others, we have the experience to take account implicitly of the various other factors/parameters that are explicitly identified in BRA.