Numbers and Metrics

Before I get to an analysis of FISMA reforms and their potential impacts, I wanted touch on something that has been biting my ass for a little while.

I get asked to justify my existence and what I provide as a return on investment. As part of that, I have tried to quantify the value of an information security or risk management program. If this were a normal blog, I would launch into a diatribe replete with all the necessary management buzz words, catch phrases and witty clichés. This is not a normal blog, so if that is what you are looking for then you can move on.

Much of the time information system risk management is based on existing risk assessment and monitoring frameworks, either insurance or financial or whatever. My opinion is that those don't work. Mainly, because those metrics are difficult for everyone to understand and they are basically rigged out to screw a customer. In the information systems world, the risk management framework needs to HELP a customer. I could be totally wrong though, but these frameworks are developed by corporations with a profit motive (not that there's anything wrong with that).

The other weak link in the chain (for me) is that even for quantitative frameworks, there is still room to fudge the numbers by a single person or data input. When you are assessing a mortgage for risk there is a history for a person, history for the house and the general climate of the market. All of which let us down in the recent past, due to securitization, greed and complicated schemes to make it all look good. There isn't a lot of room for that in the information security world.

I know what also isn't working - technical numbers. Number of viruses caught by AV, IDS true/false positive rates, percentage of environment patched. These don't work because MBA can see a 90% as pretty good. The problem with this is that even when you are at 100% there is still an unknown number of zero-day exploits in deployed software and there is still the element of human failures (lack of knowledge, misconfiguration and outright theft).

This wasn't supposed to sound all doom and gloom though. I am pointing out that somehow we as a community are doing something wrong. But please comment if you have had success in this arena. I have not seen it yet.

My idea:

I have been a fan of Eli Goldratt and the Theory of Constraints for about 10 years now, and I would love to figure out a way to apply ToC to information system risk management. You will notice I didn't say information security because a holistic risk management program includes operational and security risks.

For a couple of reasons: it is flexible for a dynamic environment, it depends on the improvement processes and the measurements for success are easy to understand.

Where's my white board?

Lame Post

This week's post is a pointer to my interview on the Southern Fried Security podcast.

http://www.southernfriedsecurity.com/the-episodes/episode-16---av-is-dead

Thanks for reading and listening.

Happy Memorial Day!

In honor of those who have and do serve our country, Happy Memorial Day!

A special shout out to Grandpa Burton who served in Europe (where he earned a purple heart during the Battle of the Bulge) and Grandpa Baker who served in the Pacific.

To my Dad and uncle who served during the Vietnam era, Semper Fi!

Last but not least my friends Mike Smith-Gulf War 1 (from EOUSA), Mike Smith-Afghanistan (from Guerilla CISO fame) and Ferguson, thanks for your more recent service.

On Greed and Complianciness

Disclaimer: This post is not inspired by any past or present events in my work or personal life. The opinions expressed here are mine and not necessarily those of any of my employers, customers, vendors or organizations with which I affiliate.

As the Gulf coast becomes another environmental disaster and still recovering from the financial meltdown, I can't help but think about the parallels between having a well run information security program and compliance. You must know by now, I am an advocate of the work NIST has done as a result of FISMA. This has led to the many 800-series documents which helped many organizations, despite what the haters may say. We need compliance and compliance frameworks, if we don't then nothing will happen.

BP was not required to have the secondary piece of equipment, so they didn't put it in. Now look what happened. Wall Street gambles with people's mortgages and livelihoods, and the taxpayers (in the form of the Federal Reserve and Bailouts) have financed the losses.

In the same vein, why would an agency or department spend taxpayer money on security when they aren't required to? Especially since there is a deficit and a push to contain costs. They wouldn't. Congress had to mandate it.

I'm not trying to make a political statement, I am saying that without compliance programs and frameworks - a company would do nothing. Without the threat of fines from compliance or public relations disasters, a corporation has no incentive to do ... anything.

So here again, let us not confuse failures because a company practices complianciness. We should also not be surprised that an organization chooses to take the path of least resistance and doesn't put resources towards a real information protection program.

Attention Cloud Fanatics

The Bureau of Engraving and Printing web site is back up. I am not sure when it came up but I thought I would conduct my own uninformed lessons learned.

My initial impressions: The cloud has the same problems that other platforms do.

I am not a cloud apologist, but I think that we can all agree that application security sucks as a general rule and not enough people are listening to OWASP.

So while I would love to throw "cloud" or outsourced services under the bus, this is an application vulnerability that could happen to any site. It is a "failure to assess" as opposed to a "failure to communicate".

There is a decent wrap-up of the whole thing here: http://www.federalnewsradio.com/index.php?nid=19&sid=1951253 My problem with that story is the last paragraphs that talk about staying patched and using anti-malware software. But at least he agrees that it isn't necessarily cloud related.

The bottom line for me is that "it's the basics, stupid". Cloud, not cloud, embedded, virtualized, whatever. It all comes back to the same types of problems and there is no easy fix.