To Pen Test or not to Pen Test .. that is the question.

I gave a Fire Talk at ShmooCon. I had hoped to convey that the way Federal agencies have been conducting their security assessments, has been flawed (at best) or wrong. I was asked a simple question "What should we do to fix it?" I gave a stock answer like document test cases better and spend more time and money on the assessment in general. It was a blow off but it was the best I could come up with while simultaneously being scared sh**less.

Then, I also was fortunate enough to be an instructor with the Potomac Forum at their Certification Accreditation Workshop with Graydon McKee and Dan Phillpott. It was truly awesome and glorious two days, but I digress. I was in the middle of a diatribe about how to assess a Federal system under the current NIST guidance and FISMA. I got to a part where I started talking about running a penetration test on the system before the accreditation/authorization to operate. Then another question "Do we NEED to run a pen test?" To which I responded ... "it depends".

Given the money and the time I would have any system I worked on penetration tested. I spent a few minutes trying to find what I mean when I say "pen test". I found it on this site, but I have it for you here:

Penetration test - A test of a network's vulnerabilities by having an authorized individual actually attempt to break into the network. The tester may undertake several methods, workarounds and "hacks" to gain entry, often initially getting through to one seemingly harmless section, and from there, attacking more sensitive areas of the network.

Who wouldn't want that? I would also add to this definition that the vulnerability is actually exploited and that evidence of the exploit it captured. Because then you have actually tested something. Running a tool and saying something like "conditions are favorable for a successful exploitation of" ... blah blah blah, is not a penetration test. That is a vulnerability assessment.

The reality is that most systems should be having vulnerability assessments done monthly, if not more frequently. Its not policy, but that's my opinion. Penetration test annually, or after substantial changes to the architecture.

Excuse 1: It Expensive.
Response: So is loosing your data.

According to a recent study (that I am currently unable to find - if you have a link then please comment), it could cost something like $200 per customer to restore their good standing. A decent test by a rock star tester is pricey. If we use this $200 number. Well - how many customers are using the system? Times Bad PR + Sleepless nights + Incident Response Services = a crap load more than the Pen Test.

Excuse 2: They could break our shhhh ... stuff
Response: We'll schedule downtime.

Most pen testers love pen testing. They also like money. Most will probably work with you to sacrifice a Saturday or Sunday evening. For one or both of these reasons. The other reality of this statement is that the person generating this excuse could be afraid of what is found. Ignorance is not bliss and obscurity is not security, the attacker will find weaknesses in the system. Get over the pride and let's just fix it.

Excuse 3: Our Coders / Developers are awesome
Response: Awesome people still make mistakes

I don't presume that most of the people who would read this trust a Bank carte blanch to handle your finances. You probably reconcile your check book, make sure that your online banking bills do, in fact, get paid, etc. Humans are not error-free neither is your code or system design. New vulnerabilities are found every day in software that we all use.

Excuse 4: We don't have time before the system needs to be live
Response: Get one after the system is live

The attackers will be working on your system from the word go. You will be required to defend it. Return to reasoning for excuse 2.

Excuse 5: Nobody wants our data
Response: You don't have a competitor?

Competitor is a wide range of possibilities. The Federal government has not just competition but real enemies. It could be another nation, terrorist or garden variety kook. If you aren't just putting information on the Internet, then clearly it requires protecting. Also, an attacker has time. Conceivably, they are motivated and they want what you have. They will spend hours, days or years working on your system and basically you need a way to outlast them.

Boss: Ok, I'm in. What's next?
You: Ahhh, yeah. I'll send you an email in the morning.

When in fact your response should have been: We are going to test everything. The guys we are bringing in can:

• Attempt compromise from the Internet;
• Attempt compromise from the inside (insider threat and/or accidents);
  
• Social Engineer our employees and service providers (that's right I said it);
• War-dial, war-drive, war-walk, war-unicycle through and around our facilities to identify unknown network entry points;
• Leave USB thumb drives in the parking lot or FedEx DVDs or CDs to insiders;
  
• (Please Comment to add more)

You may not need to do all of this each time, but it is my opinion that every organization should be going through most of these exercises on a regular basis. While they don't all fit the definition of a penetration test, these services can generally be provided by the same firm. You won't find anything in FISMA or NIST or OMB that says: "Thou shalt get a pen test", but the 800-42 says its a good idea.

So go get a pen test .. now.