I'm OK When The System Works – Even If It Is A False Alarm
———————————
UPDATE: @lbhuston gives us the dirty low down here: http://stateofsecurity.com/?p=766
———————————
This was a test of the emergency broadcast system. This was only a test, had this been a real change in the Threat Landscape…..
You may have read in various media outlets about a little incident that happened yesterday concerning the mailing of a CD full of malware to a credit union.
Before we go any further the following caveats totally apply: I’m pretty close with several of the actors in this “incident”. In fact, had this been a few years ago, there’s a good chance that I would have been the guy responsible for building the forgery and burning the CD. So my biases are apparent. And I have purposefully not talked directly with any of the parties (MicroSolved, the credit union, NCUA, SANS, ThreatPost, etc…) before sharing with you my impression and what I take away from yesterday.
So yesterday, there was an alarm raised about a “new” form of attack, purportedly against “banks” or even “the financial infrastructure” if you believed at the time what you saw on everything from national media websites to Twitter. What has been revealed so far to have really happened was this:
A credit union received a mailed a CD and letter that looked like it was from the NCUA (the gov’t body in charge of CU regulation and governance) claiming to be training materials to be viewed on a PC. But the credit union saw this as a forgery, and escalated the matter. Somehow, this attack then turned into multiple attacks on “banks” by the time it hit “big” media. An alarm went out, and basically by early afternoon, any credit union security admin who could fog a mirror knew that there might be something focused at them.
Except that it was really just part of a contracted, valid penetration test by the security firm MicroSolved. So really, it was a false alarm.
I would just like to state that I think:
THIS IS NOT A BAD THING*
Real quickly, let’s get this out of the way. For there to be a false alarm, several things must have failed yesterday. Having worked for MicroSolved, I can tell you that the paperwork we developed there for scoping is pretty durn good. When I worked there we set bounds, we described who needed to know and who didn’t, gave expectations as to attack type, general time frames to expect it, and so forth. The scoping and execution process were always phenomenal (pats self on back). But as an outsider looking at it now:
There May Have Been A Problem With The Penetration Test Scope.
This could have come from one of two sources, MicroSolved, or the CU. MicroSolved could have stepped out of bounds with scoping, or somehow unwittingly created an exception to a tight scoping process. Alternately, the CU themselves, as they were going through the scoping process, might have left out a key player on their side who is part of the fraud reporting process. I say that because:
There May Have Been A Problem With The Credit Union’s Internal Processes.
For the mailing to get to the NCUA as an actual incident, it would mean that either the credit union had poor fraud reporting processes, or someone at the CU probably didn’t follow procedure and reported to the NCUA out of process.
There May Have Been A Problem With The NCUA Alarm Process.
We don’t know what happened to cause a Pentest to be reported as an actual attack, but somewhere once the ball was handed to the NCUA, there should have probably been a verification process in place (I say this having talked last night – well, laughed is more like it- with ex-NCUA InfoSec friends). I’m guessing that there may have been a failure here, as well.
There May Have Been A Problem With The General Reporting Process.
By the time it hit SANS, SCMagazine, ThreatPost, Slashdot, The Washington Post, etc. The incident grew from one credit union to pretty much the imminent collapse of the financial infrastructure of western civilization. Again, verification and fact checking. How it got form one CU to multiple or even across the stream into banks is not known.
With That Out Of The Way…
But if we look at what happened, the time frame in which it all happened, we can see a lot of success:
- MicroSolved did the right thing in executing a feasible, clever attack.
- The Credit Union did the right thing in recognizing the attack and reporting it (even if out of process – believe me, as a veteran of Credit Union SE attacks – they could have not caught the attack or even just thrown the material away and not reported it).
- The NCUA did the right thing and got the word out.
- The Press/Media/Alerting System did the right thing and raised the alarm.
- Even we, the Security Professionals via phone to friends and via Twitter, did the right thing as a group and put the notice out.
So rather than playing a cynic and saying the system failed because a false alarm got out, I think we can say:
We Did A Pretty Good Job*
Now of course, the asterisks in both positive statements above should suggest to you my wise reader that I know that repeated false alarms are a bad thing. And there are certainly lessons learned here. But pretty much, the system of alarm worked. We did all right. And we did much better than one alternative, not sharing information about a perceived critical change in the threat landscape.
Bottom line, we shared information – and that’s pretty durn NewSchool if you ask me.
I don’t see how the fact that somebody can snail-mail malware is a change in the threat landscape, nor do I see how any real information was shared here, but I fully agree that this is much more an instance of things working properly than it is an indicator of some problem that needs to be addressed.
RE: Threat Landscape
I used “change” because I wasn’t aware of this as a common vector, and several other folks (I’ll find the links if I get a second today) mentioned that while it wasn’t conceptually new, the frequency of real attacks via snail mail hadn’t been of noticeable significance.
RE: Information Sharing
Granted, the information shared was “false”, but even that has some meaning. So while I tend to see changes in observed frequency as threat information sharing, even false alarms have meaning in terms of “certainty”.
But then again, I’m a dork with models.
It’s only a change to the threat landscape if your landscape doesn’t extend back to the pre-dot-com era.
Once upon a time in the pre-Internet days, snail mailing floppies was the only way for most of us to move data.
Even in the late 1990’s, the SEC still administered certain license exams by mailing you a floppy disk that you booted off of to take the test, then mailed back the floppy afterwards.
Nice cover for your buddies and former employer!
Sounds like they had the right idea and executed it poorly to me. If they had their Sh*t together they would have been aware of out of office dates and had appropriate contact information to nip this in the bud earlier, but still allow things to play out. I can imagine several scenarios whereby this could have run it’s course w/o setting off alarms all over.
So – well done but rather bush league.
Next time – AT LEAST know the schedules, know the alternate phone numbers. Know the escalation contact(s) at the NCUA in case things get out of control.
Next time – Perhaps provide some heads up and perhaps even identify the look of the CD to authorities and a time frame of when the “test” is going to run.
There’s a lot of ways to burn an individual or organization to prove a point – that doesn’t mean they should be utilized and if they are all parties should be very well aware of the potential consequences. – Digby
Hi @Digby!
I don’t know if it’s a “nice cover”, and FWIW I don’t work with the NCUA or SCMagazine, etc. But, I suppose you could take it that way. Frankly, I’m just working off what I saw (prior to what Microsolved is saying).
RE: Out of office “dates” I suppose that works if the PTO is scheduled. IME most OOO time isn’t scheduled (sick days, or PTO days when family needs arise). But if MicroSolved performed the test while the guy was on vacation or something, that would have been a mistake, yes. IME however, that’s something we tended to scope for (doesn’t mean there wasn’t an oversight, I’m just relating my experience).
RE: a “heads up” for the authorities, the NCUA oversees something on the order of 2,000 or so CUs that (as of several years ago at least) were “required” to have PenTests. I’m not sure that the NCUA has the infrastructure to keep up with roughly 10 PenTest notifications a day (note that spreading them out like that is a misnomer – our business used to pick up in August and not stop until February – I’m sure at least 1500 of those CUs would scedule all in Q4 to get the requirement met).
RE: “if they are all parties should be very well aware of the potential consequences.”
Kind of defeats the purpose of an SE exercise, doesn’t it? I mean, usually the process is for the client to notify only those people who need to know. If it’s a CU with a small staff, that might only be 2 people (CEO, IT Mgr). It’s a fine balance, but it’s been my experience that once you tell the wrong people, the whole organization knows. And yeah, I’ve been the SE guy on site when someone decided that they were going to let *their* employees know, and then those employees only told a friend or two and so on… It makes for a really short SE test, and is frankly a waste of about $1,500.
Digby,
If you’re going to test with live explosives, that level of effort makes sense. There’s probably less harm in this story getting out of control than others.
At what point does it become cry wolf?
Experimentally turned on comment threading. Let’s see if it helps.
Not a bad idea.
Really poor execution.
They forged correspondence from a Federal agency and then mailed it via the USPS.
The fact that it was a “valid”, “contracted” penetration test means absolutely nothing in terms of criminal liability.
The NCUA doesn’t appear to be pleased – hopefully the credit union involved didn’t have prior knowledge as to the details, otherwise an adverse action – and likely a fine is coming. Nothing like pissing the people who regulate your business off.
As for the parties involved in performing the test – I hope they have retained counsel and aren’t making comments. At the very least they’re looking at a Federal criminal investigation and a fine.
Hi @Henry,
When you say that the NCUA doesn’t appear to be pleased, is there something you’re basing that off of?
RE: the CU having prior knowledge, that’s claimed in the update link at the top of the page.
I’m basing that from the NCUA’s updated Fraud Alert.
Since the CU knew about the nature of the forged NCUA document they have a problem on their hands.
As for the consultancy involved – they’re essentially at the mercy of the Feds at this point. The fact that this is receiving more and more attention isn’t necessarily a good thing.
I’ve read Brent Huston’s spin – the fact of the matter is that NCUA and for that matter CUISPA, and the Federal Reserve have all distributed relevant and timely alerts when they become aware of an issue. This didn’t raise awareness it simply demonstrated how the system that is in place and working at the NCUA worked yet again.
All of the praise is just vibratto. These types of events are dealt with daily by financial institutions who work closely with their regulators as well as state, local, and federal law enforcement in investigating such issues.
Microsolved DID NOT DO THE RIGHT THING. Breaking the law under the guise of security testing is still breaking the law. At a minimum they’ve committed mail fraud. What’s worse is they’ve placed their client in a difficult position. I wouldn’t touch them with a 10 foot pole at this point (I wouldn’t have before either – I had never heard of them).
Meh.
It’ll depend on how the legal code is interpreted, and I’m too lazy to go rooting through statutes and case law, but I’d be willing to bet that in order to arrest or prosecute, there’d have to be considerable evidence for “intent” to defraud. It’s possible that someone could try to make a case for negligence, but IMHO that’s be specious, at best.
(IANAL, but my Sig Other is)