ISACA CRISC – A Faith-Based Initiative? Or, I Didn't Expect The Spanish Inquisition
In comments to my “Why I Don’t Like CRISC” article, Oliver writes:
CobIT allows to segregate what is called IT in analysable parts. Different Risk models apply to those parts. e.g. Information Security, Architecture, Project management. In certain areas the risk models are more mature (Infosec / Project Management) and in certain they are not (software distribution). That is for the risk modelling (sic) part.
Oliver: I’m very glad that others in our industry are preaching the concept of model selection & fit. And because you’ve demonstrated that at least you believe this is an important aspect of IRM, I’m ready to believe what you’re saying there. But before I do so, I spent a good deal of time in Missouri, so I need you to show me:
- Define “mature” – what makes a mature information risk model? In fact, show me the industry standards for gauging model maturity, so that I can examine different models, similarly.
- Show me, oh please show me, an information risk model that has even been tested (publicly) for repeatability and accuracy, more or less been shown to provide repeatability and accuracy to a measurable degree of confidence.
Now my thought is that you can’t have a mature risk model without having a measurable notion of repeatability (two analysts with the same data and same model go into separate rooms and come out with reasonably similar results) and accuracy (model outcomes have been tested to be correct some degree of the time). Maybe I’m not subscribing to the right scientific journals out there, but I’ve yet to see the data sets and the published models or model maturity tests for IRM.
For risk identification and KRIs (note to readers: I’m assuming Oliver means Key Risk Indicator – a useful but loaded phrase itself), an internal control framework which is based on cobit allows an adequate and comprehensive net of indicators for risk assessment based on operational performance.
You’re assertion is that COBIT’ is proven to be an “adequate” and “comprehensive” internal control framework. Can you show me evidence of this? What documentation for this has ISACA released? How was it proven? Where’s the study? How did they seek to falsify COBIT’s adequacy and comprehension? How was comprehensive measured? At what point was it shown that more COBIT effort decidedly into the realm of diminishing returns?
If you think that “some things can’t be measured” will prove your thesis, you don’t know Risk Management at all.
I never said that, and due to the fact that I’ve taught courses based on Hubbard’s “How To Measure Anything” to risk analysts, I’m going to offer that you don’t know me well enough to come to any conclusion about my knowledge around Information Risk Management.
What I’m saying is that ISACA, COBIT, and RiskIT aren’t mature enough to certify practitioners in a meaningful manner – where “maturity” is an ability to consistently, repeatably, and accurately show a change in risk using ISACA’s own documentation. If you can’t show me how COBIT measurably (again, where the concept of measurement requires known accuracy and repeatability – just drilling the point home, here) modifies exposure to risk or capability to manage risk in these ways, I don’t think ISACA is ready to say that we, as an industry, are more than isolated alchemists trying to find our own, individual ways to turn lead into gold. To carry the analogy, the attestation that CRISC would provide has nothing to do with knowledge of chemistry, but everything to do with the alchemists ability to repeat a known means of trying to turn lead into gold.
There is no mathematical voodoo to model a risk exposure which is 100% correct.
We’re in agreement about modeling risk exposure. To paraphrase Jaynes (poorly), probabilistic models are hypothesis and therefore we should expect (hope!) for them to be frequently falsified. In addition – just to complete the picture for you, Oliver, I’m also on record as stating that arriving at a state of knowledge for capability to manage risk is similarly difficult (and this is the whole crux of the COBIT/RISKIT/CRISC request for proof – understanding capability in a measurable way is a key dependency to understanding exposure, and therefore, ISACA is silly for trying to certify that someone can discuss exposure if they can’t even show me how COBIT reduces risk) .
You have to keep the purpose in mind and also use professional judgment based on your experience (which CRISC by the way tries to attestate)
Fascinating, so CRISC tries to provide clear evidence that an individuals experience and professional judgment is of some quality? My whole point in this series is that any individual with experience in information risk management should know enough to know that a certification around Information Risk Analysis and management is goofy. As for documenting an individual’s professional judgment skills, I’d love to see how the test does that in a rational manner.
You fight against an attestation which takes into full consideration your own challenge.
Nope. Not even close. You have no CLUE what I stand for. I’m all for good attestation. As I said the other day:
(…I’d argue that IRM shouldn’t be part of an MIS course load, rather it’s own tract with heavier influences from probability theory, history of science, complexity theory, economics, and epidemiology than, say, Engineering, Computer Science or MIS.)
My position is that given the difficult nature of risk analysis (as I’m saying above), there’s no way CRISC can attest to any competency around Information Risk Analysis, and if ISACA can’t show me how COBIT changes exposure or capability in a measurably way, then CRISC can’t possibly even attest to competency around Information Risk Management. Maybe it can serve as a RiskIT test, sure and I’m fine with that. From the same blog post as my quote above:
IRM is not (just one) “process”. Now obviously certain risk management standards (document a simple) process. In my opinion, most risk management standards are nothing BUT a re-iteration of a Plan/Do/Check/Act process. And just to be clear, I have no problems if you want to go get certified in FAIR or OCTAVE or Blahdity-Blah – I’m all for that. That shows that you’ve studied a document and can regurgitate the contents of that document, presumably on demand, and within the specific subjective perspective of those who taught you.And similarly if ISACA wants to “certify” that someone can take their RiskIT document and be a domain expert at it, groovy. Just don’t call that person “Certified in Risk and Information Systems Control™” because they’re not. They’re “Certified in our expanded P/D/C/A cycle that is yet another myopic way to list a bajillion risk scenarios in a manner you can’t possibly address before the Sun exhausts it’s supply of helium.” “TM”
I’ll state it again, if they want to change the certification’s title and meaning to simply state that an individual can do the above for RiskIT – have a day, good on you. Just don’t expect me to believe that this certification means that the individual knows anything about information risk analysis, or risk analysis in general.
ah ok, yes I missed the point
as a cissp, cisa and preparing for the cism I have to say I have no intentions on ever pursuing the new exam. the fact they are trying to grandfather people into it just says they are looking to invent something for additional revenue. grandfathering is the way to try to gain instant recognition for something built on or riding the coattails of other certs. isaca will claim they have x amount of people certified under crisc and also have a, b, c and d. so even though I more than likely have background for what they are looking for to instantly say this cert is worth something, it’s a joke that in a few months you’ll have to pay 500 (and wait the full 8 weeks if not longer) plus annual fees to get what other people got for free. the general approach just doesn’t work for me
Please re-read http://newschoolsecurity.com/2010/01/proving-crisc-is-stupid/. As stated by Don Nelson:
“Are you aware in 2002/2003 the CISM (first released) was initially offered through a grandfathering program? Per your name tag, it appears you hold the CISM???”
Does this make you not want to get the CISM? You are currently perusing this cert yet it had a grandfathering program as well. Why are you getting a certification anyway? What does the CISM actually get you? What does it mean? Do you get a raise for passing it? OR do you get recognition for the knowledge that is shows you have? Do you really have that knowledge?
A Certification can never replace practical experience. It is an added bonus to hard works that will add to there resume and build there “Search-ability” on job sites. Just as you want the CISM to show that you have practical knowledge in that area that is what the CRISC is for. I am planning to take my CISA at the end of this year… and yes I am going to apply to be grandfathered into the CRISC as I have been in IT Security for over 10 years now.
People are looking at this in the wrong light. If the CRISC means nothing then so does the rest of your certifications (and yes I hold a few; ITIL V3, SAP Security, …). I got my certifications because I wanted to show that I have knowledge in those areas. It is also a way to show that you can learn something and pass a test on it, much like a college degree. How many people that work in a business can do a double derivative right now? I know I cant!
“Just don’t expect me to believe that this certification means that the individual knows anything about information risk analysis, or risk analysis in general.”
I would never expect to believe ANY CERTIFICATION!!!! A Certification is a test with a book that people read prior to the test. Yes, I will say it again. It is a TEST that has a BOOK that will assist with the answers. For some people all the certification means is that they are good at taking a test. It does not mean they have practical knowledge of that topic. It does not mean they are qualified for the job!!! That is why there are interviews! All these certifications boil down to is to giving someone (A Recruiter) access, at a quick glance, to know that you Might have knowledge in that field. To truly find out if you do requires an interview.
People will probably deny it but truly a certification only means something when changing jobs. You will rarely get a raise in your current company just for passing a certification.
Kind Regards,
Paul
Paul is correct; certs are only good for changing jobs or contracts.
I have never been able to secure a payrise in a perminent position on the basis of having a passed a specific certification. I once tried to argue that my market rate was significantly higher due to once passing the CCSE but was ignored by my employers.
They had not paid anything for training or even the cost of the exam, although the CCSE was part of my KPIs for that year.
I took that new cert and almost tripled my income by going contracting.
I know use certifications as a marketing tool, when i bounce form contract to contract but it does allow me to get instant credibilty with the locals when they see the experaince and certs together.
I’m sure I could grandfather the cert but will most probably leave it till next year and just sit the exam and claim the CPEs for it. I refuse to sit through pod casts or webinars and prefer study and exams to get them.
As a disclaimer, I am a member of ISACA and am applying for grandfathering in the CRISC. I hold a number of vendor and vendor neutral certifications and three state professional licenses related to IT Infrastructure or Security.
Couldn’t agree more Paul. “A Certification can never replace practical experience. It is an added bonus to hard works that will add to their resume and build there “Search-ability” on job sites” This is pretty much spot on.
Presently I’m a FTE for a company but I’ve worked as a consultant most of my years in IT and Information Assurance. Previously I found that I was interviewing for a new assignment about twice a year. (My normal engagement was about six months). When I started listing certifications, and then gaining more, I found that I consistently was in the top half of the cut to interview for better and more lucrative projects. Certifications clearly helped do that.
With the exception of those of us that have to meet specific training requirements (and cortication requirements) to remain employed and access/protect/manage classified information (Read DOD employees as an example) the number one reason to certify is to have something that independently attests to general skills in a specific area and this is most crucial when you are looking for a new project or new job.
Many certs are arguably “weaker” when initially authored. It’s only with time that the certification fine tunes their exam and admission criteria. I’m not sure of a vendor program that hasn’t fit this model. One advantage of the certification is that ISACA is at least mandating continuing education and attesting to it for those holding continuous certification. This alone is a huge step up on many vendor certification programs.
Without work experience a certification is (virtually) worthless. As hiring managers we need to ensure that we are balancing “paper” with proven performance. I look at certifications (on candidate’s resumes) as evidence of a baseline of knowledge and enough dedication to their chosen career path to take the time to certify/remain certified. I weight this less than the sum of the work experience and interview performance, however if two people look similar on paper and one is certified and the other isn’t’ I’m far more likely to interview the certified individual. What I don’t do is rely on that certification instead of performing a comprehensive interview with real questions or looking for sufficient experience in the field.
Paul and Chris,
I agree, and hope that my certifications help hiring managers look at my resume of 20+ years information security experience.
Michael Thoni
CISSP, ISO 27001 ISMS PA, Archer Certified Professional, CISM, CISA, CGEIT, HITRUST Certified Professional, CRISC in progress
Available 1/1/2011 mthoni1@tamopabay.rr.com