I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 5 of 5 so please be sure to read parts 1, 2, 3, and 4.
Training, outreach, and networking
We decided to leverage existing communications avenues (mailing lists, newsletters, status reports, etc) as well as setup a Wordpress blog. We used these tools to publish information about security news, link to 3rd party security documentation, security guidance, solicit feedback and most importantly, identify individuals throughout the organization that had interest or skills in secure software development. Our goal with the outreach program was not only to make information resources available, but also to ensure that our services were transparent and accessible. Like us, most security groups have to overcome the stigma of being a crazy bunch of paranoid hackers who cost the company money and cause deadlines to slip. As such, the outreach program coupled with our threat modeling and security consulting services were delivered with clarity, transparency, and comprehensiveness.
We also delivered numerous training courses aimed at educating developers and architects in defensive programming, software vulnerabilities, and threat modeling. These courses were typically delivered to smaller audiences and accompanied with hands on activities. We found that these courses were not only being well received, but also that attendees would contact us to request additional training tailored to a specific topic that would be relevant to their products. I feel that we had the distinct benefit of having team members who were very adept at delivering training and realize that not all organizations have this sort of resources. Given a high enough priority in company goals, training can easily be purchased, and employees who are members of professional groups can leverage relationships with professionals outside the organization who would be willing to deliver the training.
For more information on homegrown security teams, checkout my post.
I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 4 of 5 so please be sure to read parts 1, 2, and 3.
Metrics and defining success
I feel that it would be easier to give you a view of the ultimate goal behind the application security metrics rather than explain how it evolved over time. The ultimate goal was for us to be able to:
Track vulnerability metrics and relevant meta data across all security assurance services.
Derive relevant data from vulnerability metrics data to aid in assigning the appropriate resources to address vulnerability trends.
Demonstrate the effectiveness, reach, and customer feedback of each security assurance service.
Provide a snap shot view of overall risk to the enterprise based on known application vulnerabilities and demonstrate trends over time.
The rational behind these goals was so that we could demonstrate the effectiveness of our services (and therefore our work), provide reliable vulnerability management data, and give us the data points needed to effectively make use of the resources we’d been allocated. I should note that we never really had an accurate picture of just how many applications were out there, so we had difficulties determining service coverage, so instead we decided to nail a stake in time as the starting point for our date based metrics.
First, we had to build a system that would collect and report on the vulnerability data points we’d identified. I’ve blogged about the system here and here where you can read the details, but to summarize, we wanted to classify the vulnerability, define risk, provide detailed description and any proof of concept data, define the product that is vulnerable, define the development team that created the vulnerable code, define contact information, and track resolution status. With this system, not only did we have each vulnerability we’d identified documented and meta-tagged, but we also had the data that allowed us to determine which development groups had the worst vulnerability rates (most vulnerabilities per product) and should be targeted for training. We also used the vulnerability data and meta-tags to provide executive level views to answer the “how many?” “what type?” “who’s doing it?” and “how bad?” questions from executives.
Findings from the penetration testing, threat modeling, and security consulting services fit very nicely into the vulnerability management system. A simple customer feedback form could have been used to rate the effectiveness of the service delivery and the derived metrics could have been used to refine our processes, we just never got that far in the project. Of course, there were several other activities which didn’t get accounted for such as training and outreach and communications who’s demand grew organically throughout the year and who’s reach in the company could have been measured over time to (hopefully) demonstrate a downward trend in vulnerability metrics for the trained development teams.
I think that the metrics plan was strong and would provide us the data we needed, I just wish we’d started implementing the full plan sooner.
Plain and simple: this book is a must read if you’re either thinking about deploying ModSecurity to protect your web applications or already have a basic ModSecurity deployment and want to learn how to customize it for your environment. Because I’m already well versed with ModSecurity, I decided read this book cover to cover without the distraction of the keyboard so that I could truly focus on how well complex topics like web application firewalls (WAF) and web application security could be covered in 250 pages. After about 5 hours of reading, I had reached the appendices and spent the next 2 days working back through the rulesets I use to protect my own web properties. Not only do I feel that I now have a more secure set of rules, I have also documented the performance impacts of ModSecurity on the Apache processes and reduced the overall response time of my applications by about 6ms.
The first 2 chapters of the book deal with installing, configuring, and rule writing basics for ModSecurity 2.5 with Apache 2.x. Magnus Mischel walks the reader through installing and configuring the module in Apache and making sure the installation is working properly. The chapter assumes best case scenarios for defaults in the operating system while in my experience, ModSecurity installs rarely go that easily. That being said, the preface of the book specifies that a good understanding of system administration principles is suggested to get the most out of the book and distribution specific installation guides for ModSecurity are widely available online. Mischel then goes on to introduce the ModSecurity rule writing syntax which he very astutely presents in two sections: rule grammar and structure followed by practical examples of how to write rules to mitigate vulnerabilities. Even after spending 1.5 years running numerous production instances of ModSecurity with custom rules, Mischel’s in depth coverage of the topic still had me hunting around for my highlighter to mark sections of the chapter containing syntax features that I didn’t know existed.
The book also does a great job of covering ModSecurity performance and logging. Mischel uses httpperf to benchmark server response time, memory usage, and processor load and presents the results for Apache with/without ModSecurity and with/without the default ruleset. Readers will find that this data will be invaluable if they ever need to sell the security benefit vs. system performance degradation to system administrators and executives. The chapter on logging and auditing offers an in depth look at the features available in ModSecurity offering insight on real world log and audit needs balanced with the processing overhead associated with excessive data collection. Mischel also provides a detailed implementation guide for deploying the ModSecurity Console with mlogc giving administrators and analysts a dashboard for monitoring alerts in a environment using multiple instances of ModSecurity.
One of the chapters I was most looking forward to reading was “Blocking Common Attacks” where Mischel presents a number of common web application vulnerabilities, offers an overview of the causes and impacts of said vulnerabilities and presents a virtual patch using a series of ModSecurity rules. To be perfectly honest, I was left a little disappointed with lack of depth in some of the content. While I understand that in depth coverage of web application vulnerabilities is probably beyond the scope of the book, the additional knowledge required to fully understand the causes and exploits of the issues that are presented in this book is likely beyond the average skill set of a system administrator. I was also left a little disappointed with the section on cross site request forgeries (CSRF) which never offered any virtual patching examples to mitigate the issue. Negatives aside, Mischel does a great job at presenting ModSecurity rules for mitigating 14 types of web application vulnerabilities such as cross site scripting (XSS), sql injection (SQLi), and shell code execution.
Mischel spends an entire chapter discussing configuring ModSecurity using a positive security model tailored to a YABB (Yet Another Bulletin Board) deployment. For a veteran ModSecurity user, this was by far the best chapter in the book, even though Mischel suggest you use Ethereal (Wireshark) as your local web proxy (you should be using Burp Suite by Portswigger). I can only hope that readers who are rolling out new ModSecurity installations or who are looking to improve existing deployments read this chapter and realize the value of this approach and choose to do the same to protect their applications.
By including the full ModSecurity directive and variable reference and a detailed guide on Regular Expressions, Magnus Mischel has created a complete guide to ModSecurity that is not only a great introduction to the WAF technology, but also a great desk reference for the experienced administrator.
Title: ModSecurity 2.5
Author: Magnus Mischel
Publisher: PACKT Publishing
Front page taglines:
* Securing your Apache installation and web applications
* Prevent web application hacking with this easy-to-use guide
I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 3 of 5 so please be sure to read parts 1 and 2.
Threat modeling, architecture reviews, and security consulting
Our application security program always offered a design review service. Initially this service wasn’t very well defined and simply involved a security engineer performing a review of available documentation and interviews with the development team. The outcome of this engagement was a series of recommendation and requirements for the project to implement during the development phase. As you can imagine, this approach was very subjective and lacked documentation, but formalization of the offering was put on the back burner due to the successes with the penetration testing offering.
A little over a year and a half after the application security program was defined, we started focusing on assimilating some of the industry proven approaches to threat modeling and security architecture best practices into our existing design review service. We leveraged the opportunities presented by the local OWASP chapter and were able to obtain the training in formal threat modeling approaches that we were easily able to adapt and offer as a fully documented and repeatable process. Unfortunately, my time with the team was up before we could really determine the effectiveness of this service offering. My hope was that we could use this service to not only identify deficiencies in secure design early in the lifecycle, but also to setup a plan for providing training, code review / SCA services, and penetration testing at the appropriate times. This would effectively setup a consulting relationship between the individual security engineers and the project team where a single person / team could be leveraged for all aspects of application security.
I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 2 of 5 so please be sure to read part 1.
Penetration Testing
Like most application security programs, ours was started by 2 of the team members who had some understanding of the vulnerability space in applications (mostly web based) and knew how to test for them. In a company with a seemingly unlimited number of custom web applications the target space was wide and varied which provided a vast playground for exploration and learning. Over time, the value of our hacking away at these web properties coupled with a few juicy findings quickly became apparent to the leadership and we were asked to work on expanding this capability to the rest of the team while maintaining the high level of thoroughness, accuracy, and professionalism our internal customers had grown accustomed to.
We were faced with the unique challenge of defining and documenting the vulnerability space, provide documented examples of how to identify and exploit the vulnerabilities, provide in person training to explore the complexities of application architectures and the interconnectedness of various components, and provide the development background necessary to work with developers to resolve the issues. Fortunately for us, the Open Web Application Security Project (OWASP) had already published some early versions of the OWASP Testing Guide and the OWASP Development Guide. These two documents provided the written documentation necessary to begin familiarizing the team with the general concepts and jargon associated with web application security. With the help of the OWASP WebGoat tool, we were able to provide a platform on which the engineers to practice what they were learning in the documents. We also conducted some additional internal training which took and in depth look at the vulnerabilities which were most common across our specific platforms and explored the best ways to address these issues in code. After a few years of regular training and refinement of the methodology has yielded a very efficient, accurate, and repeatable security testing service. If I had to go back and do it again, I really wouldn’t change anything. The resources obtained through the OWASP organization served us well and the existing internal knowledge was sufficient to bring the rest of the team up to speed.
I also wanted to address the issue of automated scanning. We did indeed purchase a solution which we worked tirelessly to implement and customize for our needs only to find that our manual approach with a couple open source tools was performing much better than the automated solution. This was partially due to the application we were testing, we had some particularly unique extremes in size of the application to be tested, and the automated tools simply didn’t complete scans in an amount of time that we found acceptable. If I had to do it again, I would explore some more varied options for automated testing that could be better fit for the environment. I realize the importance of an automated solution to validate applications which do have strong security requirements, but that do require some level of security assurance.
I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. Since this promises to be a pretty long post, I’m going to break it up into more digestible bits containing an initiative or two.
Dedicated application security team
When I was first hired on to the team, it was responsible for both security assurance (application security) and incident response. One of the first milestones which lead to the success of the program was the separation of the team into 2 groups with distinctly different areas of responsibility: one team did security assurance, the other did incident response. I credit this move as a success because when the responsibilities were shared, engineers were required to share their focus between the proactive mindset of security assurance, and the reactive mindset of incident response. Inevitably, certain engineers were more skilled in one or the other of the disciplines and the quality of service across the different offerings would be degraded at times. While the teams now had a different and separate structures and managers, the importance of information sharing wasn’t lost as the teams were already so used to collaborating as a single team.
If your company is serious about application security and the workload justifies it, I would suggest that a dedicated application security team be created to provide the expertise needed to address the unique challenges of a complete application security problem space. The charter and team responsibilities must be carefully selected, if the mission is too narrow certain critical of the supporting components in application architectures might be out of scope, while a mission that is too broad will reduce the amount of time engineers will be able to dedicate to appsec will be limited. If I had to do it over again, I would certainly narrow the scope of the team to only include application security and associated disciplines (training, threat modeling, secure design and architecture consulting, security assessments, code reviews and analysis – SCA).
In this time of shrinking budgets, reduced staff, and other various financial constraints, security departments world wide are looking for ways to justify the expense of a well rounded application security program. Jeremiah Grossman (WhiteHat Security / OWASP) and Jeff Williams (Aspect Security / OWASP) have collaborated on a fantastic article that is sure to get security engineers, developers, and managers alike lining up at their executives’ door armed with some great arguments in support of creating or expanding their application security programs.
While I fully support and endorse the content of the article, I can already see the responses from some executives:
“Wow, this is really great, and I fully buy into it, but as it stands, there is REALLY no money extra money in the budget.”
What now? Hopefully, there are employees in your company who are responsible for managing servers, the domain, the networks/firewalls/IDS/IPS, and development efforts; but most importantly, you, who is concerned about the state of application security. Why not use some of the best resources from those groups to form an application security working group? I’ve found that no matter what the company, there are ALWAYS people that are interested in being able to devote some time to learning about how those sneaky hackers are able to break into so many systems. Furthermore, what (good) technologist wouldn’t be interested in expanding their skills?
I’m sure that you’ll encounter some resistance from employees and management alike, after all, time is valuable, but remember, Jeremiah and Jeff have already given you all the ammo you need; use it wisely.
Now that you’ve identified folks who are willing to donate some of their valuable time to this effort, your first step will be educating them about the various activities and processes involved with a complete application security program. Here’s a quick reminder:
Design and architecture reviews and threat models where you work to identify flaws in the application design and architecture as well as imposing security requirements.
Secure coding standards where you work to ensure that developers implement features such as input validation, authentication, authorization, and avoid creating flaws in the business logic as they move through implementation.
Code reviews where through either automated or manual means, developers review their code with a focus on identifying any security issues.
Testing and validation where you ensure that all requirements have been implemented according to design specs.
Deployment and maintenance where you ensure that logs are being monitored, system patches are being kept up to date, and 3rd party libraries used in the application are kept up to date.
The chances are good that some of the folks that have chosen to join the security working group already have some valuable input to provide. For example, the sysadmins might have expertise in Apache, host configuration management, or IPTables. The network folks might have been exposed to web application firewall technology. The developers might have some great security specific libraries that they’ve used on past projects. All of this knowledge is valuable and already at your finger tips. For other topics, search around your social network, if you’re on Twitter, checkout @securitytwits, if you’re a member of OWASP, attend the meetings, talk to folks, try to get an idea of their expertise. Many of the seasoned security professionals who attend these meeting would be happy to come in and present at one of your groups meeting.
So, get out there, talk to people both within and outside of your company, not only will you expand your social network and improve your companies application security program, but you’ll also be giving new opportunities to those who join the security working group.
Mubix recently had a blog post where he detailed an approach for building a team to defend the infrastructure in a hacking contest. This got me thinking about how I might recommend going about building an enterprise security team.
Know that you are the right person for the job – Building a team of security professionals requires a strong understanding of many different areas of specialization in the computer field. You must not only have the people and business skills typically required in any other leadership role, but you must also have a deep understanding of technologies (both security and other) and how they inter operate to ensure reasonable protections against disasters. I’m not saying that you must be able to configure the PIX firewall, but you should understand it’s purpose, differences in firewall technologies, their challenges and how they play into the overall security architecture.
Know your mission – the first item of business is to determine what the security team is going to provide in terms of services for the enterprise. What are the problems that you were asked to solve by creating a security team? Some examples might include PCI/SOX/HIPAA compliance. Perhaps a savvy CEO has asked you to build enterprise security guidance in the form of policies, standards and baselines. Maybe you’ve inherited the network firewall ACL management, penetration testing, enterprise authentication management and product security design services to better serve the company’s need for IT Security.
Know your infrastructure – now that you’ve determined all aspects of the enterprise that are going to be under your control, it’s time to audit them and determine whether any security controls are already in place. Remember, you’re going to need to build the team to support and enhance these controls, this exercise will help you determine a required skill set. This is also the point at which you’re going to audit the parts of the enterprise you’re being tasked to protect. It is important to involve all stakeholders so that a single set of requirements can be derived for each of the security services. For example, if one of your tasks is to evaluate and standardize security controls over all production hosts, it’s important to determine the operational needs of the system administrators who manage these hosts and the developers who write the applications running on these hosts.
Team leads – Here’s where the size of the enterprise you’ve been tasked with protecting comes in. If you’re a small company, your “leadership team” might be 3 really talented individuals with whom you plan on dividing the responsibilities. On the other end of the spectrum, your leadership team might be 5 managers with teams of engineers to accomplish the work in a large enterprise environment. The people in your leadership team (whether the enterprise is big or small) must be colleagues that you trust will not only be able to help you execute your vision, but will also be able to provide valuable input to the plan. The leadership team must all share the same general understanding of enterprise security architecture components while maintaining a strong subject matter expertise in the specific technology areas which they are directly responsible for. For example, I would expect that your network team leader could not only lead the engineers on the team, but also be able to step up to the keyboard and fill an engineer’s shoes.
The engineering teams – Provided you’ve been given enough budget and are looking to secure a large enterprise, you’ll be looking to fill at least a couple vacancies on each team. The interviews should be conducted such that the candidate is not only demonstrating knowledge in the areas directly associated with the position they’re applying for but also in the other areas of responsibility for the security team. In addition to the usual traits you look for in a good candidate (good resume, demonstrated technical knowledge, good writing/communication skills, overall fun person), you are also looking for candidates who are not only experts in their fields, but possess a broad knowledge of technology in general. Don’t be afraid to conduct several interviews with several different people and keep the interviews challenging by presenting a real life security concern and have the candidate and interviewers solve it as a team.
Solve the problem on paper – Now that you have your awesome security team in place, it’s time to get them all together and lay out the problems you’ve been tasked with solving. This is when your vision for solving the problems gets mulled over and revised by your trusted colleagues. This exercise should not only add specific technology solutions to the plan, but will also allow everyone the opportunity to apply their knowledge and be invested in the final solutions.
Measure it – In any enterprise, progress must be measured to ensure some form of Return on Investment (ROI). Costs associated with technology purchases, staff salaries, and other activities will have to be be justified by demonstrating that the cost provides a reasonable assurance of protection against threats. Additionally, in the security world, a lot of useful planning information can be derived from tracking security metrics. For example, if the security assurance team is noticing a high number of input based vulnerabilities during their application assessments, perhaps some investment in secure development training for the dev teams is in order. Knowing which metrics to track as the plan is being built will facilitate the collection and tracking of the numbers.
I do realize this is pretty high level and lacking in specifics, but I wanted to be sure I highlighted the importance of shared technical security knowledge as a key factor in the success of the security plan and subsequent implementation. Furthermore, I beleieve that the same planning and executing techniques can be applied from the executive goals to the engineers’ individual goals and when given the chance to build those goals together, the likelyhood of a good plan being build and implemented increases greatly.
In order to effectively implement my plan for tracking enterprise vulnerability metrics and their associated data points, it was clear that a custom application for data entry and report viewing would be required. As such, I wrote a PHP/MySQL web application with the following features and functionality.
Data Model – In order to structure the collected data in the database in a manner which facilitates data extraction from reports, I had to define the reporting needs early in the development process. By doing so, I was able to create a relational data model which makes reporting fast, accurate, and reliable.
Data Input – I’ve provided a series of HTML forms which provide the user with all the data points from my previous article. These forms have been designed with a heavy focus on ease of use and rapid data entry. To provide these features, the input for the required data points have been grouped logically and implement a feature for quick input where the user can select from a list of recent entries for the given data points (eg: recent contacts, recent CVE numbers, etc).
Reporting – I’m a big fan of keeping reporting as open as possible: it makes life much easier down the road. As such, I created a reporting framework which is capable of outputting a list of vulnerabilities based on a query for any data point stored in the database. For example, if you want to know which reports are associated with a given manager, business unit, vulnerability class, or keyword just select your preferences on the search page, and view the results.
Statistics – A data collection tool is worthless unless you have a way to quickly determine the characteristics of the data stored in the database. As such, I created a statistics page which provides the viewer with a snapshot of the vulnerability metrics based on similar characteristics. The following details the statistics and their intended purpose:
Vulnerabilities Created and Resolved
Simple count of vulnerabilities created and resolved (separate counts) in the system for the given time period.
Vulnerabilities Created and Resolved per Category
Simple count of vulnerabilities created and resolved (separate counts) in the system for the given time period per vulnerability category.
Vulnerabilities Created and Resolved per System Type
Simple count of vulnerabilities created and resolved (separate counts) in the system for the given time period per system type.
Vulnerabilities Created and Resolved per Source
Simple count of vulnerabilities created and resolved (separate counts) in the system for the given time period per vulnerability source.
Vulnerabilities Created and Resolved per Impact Rating
Simple count of vulnerabilities created and resolved (separate counts) in the system for the given time period per impact rating category and value.
Vulnerabilities Created and Resolved per Risk, Severity, and Probability
Simple count of vulnerabilities created and resolved (seperate counts) in the system for the given time period per risk, severity, and probability rating.
Vulnerabilities Created and Resolved per Domain
Simple count of vulnerabilities created and resolved (seperate counts) in the system for the given time period per domain name.
Top Report Owners
Simple count of vulnerabilites created and resolved with average time to resolve in the given time period per report owner (the security engineer assigned to the vulnerability).
Top Business Units
Simple count of vulnerabilites created and resolved in the given time period per business unit listed in the contact field.
Top Managers
Simple count of vulnerabilities created and resolved in the given time period manager listed in the contact field.
The tool also contains several usability features such as mass actioning of vulnerabilities (eg: resolve multiple reports at once), a data authorization model defined by report owner (owner can edit and view, others can only view), and a “my” page which displays reports relevant to the user.
In the future I hope to seperate the parts of the tool which are specific to my environment and open source the project so that others can gain visibility into their vulnerability posture.
Some time back I posted about designing and implementing the processes and tools which would enable me to track vulnerabilities affecting the enterprise throughout their security life cycle (identification to mitigation validation). In addition to providing a central repository for all enterprise vulnerabilities, I also wanted to make sure that we could use the metrics derived from this data to support investments in other areas of IT Security. My target was to have everything ready to start tracking vulnerabilities on the first day of business in 2009.
The data points I decided to track were primarily aimed at providing visibility into how vulnerabilities were identified, who was responsible for them, establishing an enterprise risk profile, and categorizing vulnerabilities according to risk factors. The following is a more detailed list of the metrics I’m tracking and their intended purpose.
The Data Points
Report Source – Tracking where the vulnerability reports are originating from. As options, I decided on direct report, findings from automated scanning, and all relevant existing security processes (security assurance, incident response investigation, etc) as sources.
System Type – Tracking the type of system affected by the vulnerability. As options, web applications, desktop applications, hosts/networks/operating systems, and web platforms seem to encompass all major system types.
Vulnerability Class – Being able to abstract individual vulnerabilities into something that is digestible by a non-technical audience is critical in this field. The National Vulnerability Database (NVD) has adopted a cross section of the Common Weakness Enumeration (CWE) hierarchy that is broad enough for the diversity of vulnerabilities most enterprises would encounter. For more information, checkout the CWE page on the NVD website, http://nvd.nist.gov/cwe.cfm.
Impact Ratings - Giving us a smart way to answer the “how bad is it?” question. The NVD has developed a framework (Common Vulnerability Scoring System – CVSS) where the characteristics of the vulnerability are supplied and the high, medium, low ratings for risk, severity, and impact are derived mathematically based on the supplied characteristics. The mathematical formula for the computation is available in the CVSS documentation, but I had a different idea for computing the h/m/l ratings. First, here are a few of the characteristics, their ratings, and descriptions:
Access Vector (AV)
Local - vulnerability can only be exploited from the local system. LAN - vulnerability can only be exploited from the local area network. Network - vulnerability can be exploited from the internet.
Authentication (AU)
User - User level authentication is required to exploit the vulnerability. Admin/Root – Administrator/root level authentication is required to exploit the vulnerability. None - No authentication is required to exploit the vulnerability.
Remediation Level (R)
Official Fix – The vulnerability has been fully remidiated and tested according to guidance given by security staff. Temporary Fix – The vulnerability has been mitigated temporarily while a permanent solution is devised. Workaround – A workaround exists to circumvent the vulnerability. Unavailable – No remidiation is available for the vulnerability.
Not Defined – Remediation for the vulnerability is unknown.
Additional impact ratings include Access Complexity (AC), Confidentiality Impact (C), Integrity Impact (I), Availability Impact (A), Exploitability (E), and Report Confidence (RC). Each of the values associated with the impact ratings is assigned a whole integer from 1 – 3. For impact ratings where there are more then 3 options, I found that in each case, 2 of the options were close enough to be assigned the same value. For determining which value I would assign to each impact rating, I assumed the greater the value of the impact rating, the more serious the vulnerability for the given impact. For example, for Authentication, User is assigned a value of 2, Admin/Root a value of 1, and None a value of 3. I then weighted each impact rating based on the importance attributed to the impact. For example Access Vector (weight of 2) is much more of a factor when determning risk than Report Confidence (weight of 1). To further clarify the impact of a vulnerability, I wanted to produce ratings for risk, severity and probability which are determined with the following equations:
Risk: (2*(av) + 2*(ac) + 2*(au) + c + i + a + 2*(e) + 4*(r) + rc)/15. Severity: (c + i + a + 2*(e) + rc)/6. Probability: (2*(av) + 2*(ac) + 2*(au) + 4*(r))/10.
Once the risk, severity and probability have been determined, I use the following ranges to assign high, medium, and low:
High: >= 2.00. Medium: >= 1.00 and <= 1.99. Low: >= 0 and <= .99.
Contact Information – Tracking who is reporting the issues, and who is responsible for fixing them. For each contact entered into the system, I wanted to capture their name, contact information, business unit (only for enterprise contacts), and their manager (again, only for enterprise contacts). The purpose of the contact information data points is to be able to determine which group within the enterprise we can target for education regarding specific topics in security. For example if we find that product team X has an abnormally high number of input validation vulnerabilities, we could justify investing in some XSS and SQLi training and know that it will be beneficial.
URL – I wanted to be able to present the data by fully qualified domain name (FQDN) to easily present the data to site owners.
Other - In addition to the data points mentioned above, I’ve also provided inputs for tracking of CVE numbers and internal ticket tracking system numbers. While this isn’t particularly relevant to vulnerability tracking, it will give my management a tie in to existing processes.
Provided that the data entered into the vulnerability tracking system is reliable and up to date, answering the “What’s wrong?“, “How widespread?“, “Who’s responsible“, “What’s the exposure?“, “What if we don’t fix it?“, “Where should we invest resources?“, and “How’s the security?” questions should be no problem from here on out.
I’ve spent the past 3.5 years working on a team where my primary responsibilities involved “application security”. Now that this era has come to an end, I’d like to share some of the initiatives and define their successes and shortcomings. This is part 5 of 5 so please be sure to read parts 1, 2, 3, and 4.
DandyID
Friendfeed
claimID
Disqus
Facebook
Twitter
myOpenID
Google Profiles
Linkedin