Blog :: by Wade Woolwine

Thoughts on an AppSec Program (Pt. 2) – Penetration Testing

I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 2 of 5 so please be sure to read part 1.

Penetration Testing
Like most application security programs, ours was started by 2 of the team members who had some understanding of the vulnerability space in applications (mostly web based) and knew how to test for them. In a company with a seemingly unlimited number of custom web applications the target space was wide and varied which provided a vast playground for exploration and learning. Over time, the value of our hacking away at these web properties coupled with a few juicy findings quickly became apparent to the leadership and we were asked to work on expanding this capability to the rest of the team while maintaining the high level of thoroughness, accuracy, and professionalism our internal customers had grown accustomed to.

We were faced with the unique challenge of defining and documenting the vulnerability space, provide documented examples of how to identify and exploit the vulnerabilities, provide in person training to explore the complexities of application architectures and the interconnectedness of various components, and provide the development background necessary to work with developers to resolve the issues. Fortunately for us, the Open Web Application Security Project (OWASP) had already published some early versions of the OWASP Testing Guide and the OWASP Development Guide. These two documents provided the written documentation necessary to begin familiarizing the team with the general concepts and jargon associated with web application security. With the help of the OWASP WebGoat tool, we were able to provide a platform on which the engineers to practice what they were learning in the documents. We also conducted some additional internal training which took and in depth look at the vulnerabilities which were most common across our specific platforms and explored the best ways to address these issues in code. After a few years of regular training and refinement of the methodology has yielded a very efficient, accurate, and repeatable security testing service. If I had to go back and do it again, I really wouldn’t change anything. The resources obtained through the OWASP organization served us well and the existing internal knowledge was sufficient to bring the rest of the team up to speed.

I also wanted to address the issue of automated scanning. We did indeed purchase a solution which we worked tirelessly to implement and customize for our needs only to find that our manual approach with a couple open source tools was performing much better than the automated solution. This was partially due to the application we were testing, we had some particularly unique extremes in size of the application to be tested, and the automated tools simply didn’t complete scans in an amount of time that we found acceptable. If I had to do it again, I would explore some more varied options for automated testing that could be better fit for the environment. I realize the importance of an automated solution to validate applications which do have strong security requirements, but that do require some level of security assurance.

Tags: Enterprise Security, Hacking, OWASP, Penetration Testing, tools, WebAppSec