Shostack + Friends Blog Archive

2010 Security Prognosticators – Put Your Money Where Your Mouth Is!!!

Just saw where Symantec has released their 2010 Security Trends to watch.  Now not to pick on Symantec (I’m guilty of the same mess in the past myself over on my old blog) but usually these sorts of prognostication lists are full of the same horse@!@#$.  For example:

8.  Mac and Mobile Malware Will Increase
In 2009, Macs and smartphones will be targeted more by malware authors. As Mac and smartphones continue to increase in popularity in 2010,

“More” is a fuzzy, useless prediction.  We have a fairly benign “DNS Changer” thing on the Mac.  And that’s about it (source: an informal and utterly unscientific poll of College Security Admins I did on Twitter).  Does “more” = something else you have to be looking at naughty pr0n and give admin rights to be taken advantage of?  Or does it mean something that will cause us all to actually *use* anti-malware on the Mac?  We don’t know.  But all the author needs is “more = another”, and they’re right.  Bleh.

TURN THE BEAT AROUND

So this year, let me challenge you to make a change.  If you think that there’s going to be a “trend” or “something” to watch for in 2010, let’s see you put your money where your mouth is and be specific.

By specific, I mean go ahead, play weatherman and add an 1-100& “chance” that it’ll happen.   What I’ll do here on the NewSchool blog is collect these, and then we’ll do an ad-hoc sort of “Alex + Brier Score” model on the foretelling this time next year and we’ll see who does a good job.  Yep, it’s a challenge – if you think you’re good/important/wise enough to make a prediction for next year, then you don’t mind if we hold you accountable, right?

Score Rules/ Model:

1.)  We’ll use Wikipedia’s Brier Score example as the basis for our Model:

Suppose it is required to give a probability P forecast of a binary event – such as a forecast of rain. The forecast issued says that there is a probability P that the event will occur. Let X = 1 if the event occurs and X = 0 if it doesn’t.  Then the Brier score is given by:

• If you forecast 100% (P = 1) and there is at least 1 mm of rain in the bucket, your Brier Score is 0, or “perfect”.
• If you forecast 100% P and there is no rain in the bucket, your Brier Score is 1, or “awful”.
• If you forecast 70% P and there is at least 1 mm of rain in the bucket, your Brier Score is (0.70-1)^2 = 0.09, or “not too shabby”.
• If you forecast 30% P and there is at least 1 mm of rain in the bucket, your Brier Score is (0.30-1)^2 = 0.49, or “needs work”.
• If you hedge your forecast with a 50% P and whether or not there is at least 1 mm of rain in the bucket, your Brier Score is 0.25, or “no courage”.

Then I’ll poll NewSchool bloggers to see if the prognostication was “lame” (i.e. the sun will shine at some point in 2010).  I’ll use an ad-hoc completely stupid 1-10 scoring system where 1=lame and 10=gutsy, multiplying the Brier score by the “Alex” score to come up with the final score for the prediction.

Just to make this even more fun, in addition, we’ll also gather a “% cowardly prognostication” metric.  The losers will be given the “Brave Sir Robin” award for soiling their armor in the face of a cute little bunny.

Originally published by alex on 30 Nov 2009
Last modified on 30 Nov 2009
Categories:   Uncategorized