The lumbering ogre of Enterprise Governance is no replacement for real Quality Management.
Gideon Rasmussen, CISSP, CISA, CISM, CIPP, writes in his latest blog post (http://www.gideonrasmussen.com/article-22.html) about the BP Oil spill and operational risk, and the damages the spill is causing BP. Ignoring the hindsight bias of the article here…
“This oil spill is a classic example of a black swan (events with the potential for severe impact to business and a low rate of occurrence)[vi].”
No. No it’s not. A Black Swan is something for which our prior distributions are completely uninformative. In this case there was plenty of prior information about Deepwater, both from a “macro-analytical” standpoint (frequency & impact of oil well accidents) and from a “micro-analytical” standpoint (there were plenty of warnings about mis-management leading right up to the spill).
Now some of you readers will be thinking “there goes Alex again, waging war against Taleb’s stupid mischaracterization of ‘black swan'” and yes, Gideon is using “black swan” when he means “tail event” – I don’t blame him for that, it’s a common error perpetuated by that awful book. Bear with me….
That’s not my point today. What is important is this:
We (the risk & data analysts of the world) need to be really careful about how we’re communicating to management. Saying that Deepwater was a “Black Swan” or more properly, a “tail event” can allow someone to think that they just got “unlucky”. This is crap. BP did not get unlucky, they got cheap, lazy, and sloppy. And not just at the well, either. If (and this is just an “if”) upper management’s tolerance for risk was NOT reflected by the singular judgement calls made to circumvent appropriate safety controls, then upper management suffered what some would call a “governance” problem (I use the term very begrudgingly here – more on that in a bit), and a significant one at that. And since rant mode is on, let me explain that this is one thing that bugs me about IT or Op risk assessments – the impact of organizational behavior is rarely taken into account. Take, for example, R=TxVxI (please?). “V” is not just the weakness in the system we see, it is a cocktail of operational skills, resources, management (don’t make me say governance here, please), and yes, even “luck”.
SO the lesson here might just be that risk communication (and before you go there, IHMO COSO is self-defeating – see below) is a significant part of the risk analysis determination. We security people focus on “upwards” communication of risk – trying to educate C-levels about the dangers they face. But I’d bet that if an organization is incapable of communicating tolerance effectively from the top down, then they are likely to have more problems than those that don’t. There can be a time-lapse problem (Jaynesian entropy if I can use that term) between the operational happenings (what’s going on at the well) and the ability of those ultimately accountable (sr. mgmt) to detect, respond, and prevent risk issues from happening.
Even worse? We’re keen on adding more bureaucracy to solve the communications problem in the name of “recognizing” and “managing” risk (GRC, ERM councils, Legal departments, bleh). But in an organization the size of BP, a “GRC Dashboard” just isn’t going to solve the “micro-analytical” problems faced at Deepwater (assuming that BP executive management would have had a lower tolerance for probable incidents than the decision makers at the well).
The lumbering ogre of Enterprise Governance is no replacement for real quality management.
One can only imagine if BP had an Operational Risk Program like our standards and consultants tell us we should be operating. What are the chances that the problems at the well would have been politically covered up, or been part of a 24 month “Enterprise Risk Assessment”, with Deepwater’s issues being one of (hundreds of?) thousands of individual risk issues documented very nicely and expensively, but never effectively communicated to the board?
There has GOT to be a better way.
Looks to me like the message about risk tolerance was received loud and clear.
“The management” maximized short-term shareholder value. Bummer for everybody else. If you don’t like that maximand, then I think “governance” is indeed the way to fix it – the lawmaking kind.
Well, I don’t hate the Taleb’s book, but I totally hate such blatant delusions as “a black swan (events with the potential for severe impact to business and a low rate of occurrence)”
Thanks for setting the setting the record straight. I think we in security need a nice and clear (well, as clear as possible) separation between:
– rare due to a combination of HUGE stupidity with HUGE luck – well, until that moment 🙂
– rare due to complete, inherent unpredctability (black swan)
Awesome post. Love to hear people tear down the industry accepted terminology.
I agree that governance and quality management is key. Corporate Governance and IT Governance is at least well understood and has issuing frameworks, even if you don’t like some of them (COSO, which I do — but needs to be combined with IT governance as well).
For IT and Ops, data and system quality management is not widely understood or commonly practiced. The Gartner Infrastructure Maturity Model is better, IMO, than ISO 20k / ITIL in terms of systems quality management. However, there is a huge lack of system quality management in general… network management and service management takes a precedent.
Worse, there is nothing out there for data quality management. I’ve seen about one book on the subject and this was very recent. Actual implementations of data quality management are non-existent. How can we expect data security (and application security) without data quality management? Also, it appears that you have misplaced appdev’s and SQE’s role in application quality management as well — not many organizations operate solely via COTS and there is always a high degree of application customization that is mired in appdev.
Also missing for orgs is Data and AppDev Governance. Would these map directly to IT Governance or fall under its umbrella? Should they?
Enterprise Governance that includes these elements does appear to be on Gartner’s radar (or would that be Forrester who markets their superior radar?) in their form of Enterprise Security Intelligence (ESI), but it appears too tactical and operational without being very strategic. GRC dashboards must be cross-organizational and account for silos by listing the on-going current implementations of controls/mitigations alongside the preferred controls/mitigations as well as the optimal controls/mitigations. They must influence (and be influenced by) actual governance (as you describe — although I would add Enterprise architecture and capital planning) as well as have cross-discipline tactical controls that have proven to be both sufficient and efficient to prevent and reduce the in-scope risks.
Even so, for me a Black Swan aspect of the incident has been the subsequent reputational damage to BP. This has not been a trial by public media, but trial by social media and ultimately, trial by PageRank. In web 2.0 there is no such thing as yesterday’s news, or yesterday’s newspapers wrapping up today’s fish and chips. Links are just as good today as they were yesterday, and continue to remain search-worthy far into the future as long as PageRank deems them to be so. Holding steady at approximately two thirds of the search market, Google via PageRank has become the default arbiter of Internet truth.
http://lukenotricks.blogspot.com/2010/09/bp-and-trial-by-pagerank.html
Guys – just found the blog…loving the rants; but more so the fact that at least some people in this industry of ours have the capability and will to apply critical thinking to what is blatently a canon of self perpetuating delusions and inaccuracies.