Application Security Guide For CISOs

Introduction

The aim of this project is to help Chief Information Security Officers (CISO) in establishing and managing an application security program that addresses the different application security goals of the organization such as meeting the information security compliance requirements and reduce the risks to the business due to the cyber threats seeking to exploit web application vulnerabilities. Because of the role that CISOs play in the organization, CISOs are critical in deciding what policies and governance are required to security operate the business within their business domain. Since CISOs are responsible of information security and governance, they are also responsible for application security governance, application security risk management, incident management, prioritization of security measures and security investments including application security measures such as processes, people and information technologies.

In this guide a particular emphasis is given to analysis of the impacts of security incidents caused by attacks against web applications. Due to the evolving threat landscape that seeks to target web applications as the main target, CISO are challenged to invest in application security measures to mitigate the risks of these threats. The aim of this guide is to provide guidance to CISO for prioritize the investment in application security measures by considering criteria such as the quantification of risk and the monetization of the impacts of data breaches to the organization. This impact is compared with the benefits of investment in application security measures and activities. From security risk management perspective, the main focus of this guide is on mitigating the risk of application vulnerabilities might severely and negatively impact the business. Examples of negative impacts consist on the increased costs of recovering from application security incident causing data losses, online fraud, loss of revenue and reputational damage to the organization.

Besides investing in detecting and fixing web application vulnerabilities, CISOs today need to also invest in new countermeasures to mitigate the risks of new threats. Besides security controls and technologies, application security measures as a whole play an important factor in mitigating the risks to the organization. The planning of application security measures need to take into consideration the maturity of the organization in security governance and security risk management processes. This will allow the CISO to plan for the roll out of application security activities that can evolve from an ad-hoc application security activity to several others that are standardized and consistently managed across the organization. These include application security processes/tools such as architectural risk analysis/ threat modeling, secure code reviews/static source code analysis and application security testing/web application vulnerability scanning. This guide aim to help CISOs in achieving compliance with security standards and regulations, reduce risks of application vulnerabilities and prioritize security investments where is most cost and risk mitigation effective.

A reference to the several OWASP resources is provided that includes application security guidelines, security training modules and security testing tools. Among the CISO goals for application security, meeting compliance with information security policies is often the one that has the most focus. The aim of this guide is shift the focus of CISO from security compliance to the reduction of security risks posed by threats and vulnerabilities of web applications. Since investement in compliance as well as operations risk management are CISO responsabilities, the focus of investement in application security measures should be on what is most cost effective to manage risks. After the investments are made, it is important for CISO to report on the status of both appliction security governance and application security risks management as well as to make informed decisions on how to manage application security risks more effectively.

Goals

In summary, this guide is being developed and structured to achieve the following goals:

1. Create the business cases for application security
2. Provide the criteria for what and where to target spending for application security measures, processes and technologies/tools
3. Link to OWASP application security resources (projects, guides, tools)
4. Guide on application security metrics for measuring compliance and cost effective managing of the risk of application

Structure

This guide is articulated in four parts:

• Part I provides business cases & risk-cost criteria for application security spending;
• Part II provides application security issues that can be prioritized and targeted for risk mitigation;
• Part III provides guidance on which application security processes and activities can be targeted for spending. A reference to OWASP projects will be also provided as resources that can be leveraged;
• Part IV provide examples of metrics and measurements for vulnerability and risk management

Foreword

This project is being developed by the OWASP Global Industry Committee in parallel with the CISO Survey project.
The objective is to run these projects in synch and use the results of the CISO survey to tailor the guide to the specific CISOs needs by highlighting which OWASP projects/resources address these needs.
The draft of the current guide is included herein:

Part I: Business Case & Risk-Cost Criteria for Application Security Spending

Introduction

In the digital era, global organizations serve an increasing number of customers through online web and mobile software applications. Several of these web applications provide highly trusted services to customers, in the case of financial services for example, these include feature-risk services to open bank accounts, pay bills, apply for loans, book resources and services, transfer funds, trade stocks, view account information, download statements etc. This online experience is convenient for individuals: it allows them to perform the same financial transactions as being at the branch/office/outlet, but with the added convenience of conducting these transactions remotely from their home computer or mobile phone. At the same time, this convenience for customers comes at a price to the organizations involved in developing and maintaining them. Online banking and commerce sites for example have become the target of an increased number of cyber-attacks and incidents. Several of these incidents resulted in a denial of online access, breaches of customer’s data and online fraud.

In the case of data breach incidents, often these attacks involve the exploitation of vulnerabilities in the applications such as cross site scripting and SQL injection. The target of these attacks is the data assets that the site stores as well as the business transactions provided by the applications that processes these data. In the case of online banking applications, the data targeted by hacking and malware include personal data of customers, bank account data, credit and debit card data, online credentials such as passwords and PINs and last but not least, alteration of data in on-line financial transactions such as transfers of money to commit fraud. Verizon’s 2011 data breach investigations report (Ref [1]) identifies hacking and malware as the most prominent types of attack, yielding stolen passwords and credentials, and thus posing a major threat to any organization that trades online.

To cope with this increase of incidents targeting web applications such as denial of services and data breaches often caused by hacking and malware, Chief Information Security Officers (CISOs) have been called by senior executives in their organizations to roll out application security measures to avoid, mitigate and reduce these risks to the organization. The increasing threat to web applications such as online banking applications challenges CISOs to consider an increased investment in application security to cope with the increasing threat.

In the last decade, investments in application security have been a growing proportion of overall information security and information technology budgets, the 2009 OWASP Security Spending Benchmarks Project Report (Ref[26]) for example states "Despite the economic downturn, over a quarter of respondents expected web application security spending to increase in 2009 and 36% expected to remain flat". Nevertheless, making the business case for increasing the budget for application security today still represents a challenge, especially when competing with the same budget allocated for application development of new features, support of new devices such as mobile phones and investments to retain and attract new customers as well as expand the service uptake or profitability.

Ultimately, in today’s recession type of economic climate and in a scenario of slow growth in business investments including the company’s built-in software (Ref [2]), it is increasingly important for CISOs to articulate the "business case" for investment in application security. Since it also appears to be a disconnect between organization's perceived threats (application security threats are greatest) yet spending on network and infrastructure security is still much higher (Ref[27]) we would like to shed some light on the business impact of data breaches due to application vulnerability exploits and how much these might costs to organizations.

Typically, additional budget allocation for application security includes the development of changes in the application to fix the causes of the incident (e.g fixing vulnerabilities) as well as roll out of additional security measures such as preventive and detective controls for mitigating risks of hacking and malware and limiting the likelihood and impact of future data breach incidents. CISOs can build a business case for additional budget for application security today for different reasons; some directly tailored to the specific company risk culture or appetite for risk; others tailored to application security needs identified in application security surveys such as OWASP Application Security Survey. Specifically, (TBC based by the OWASP CISO survey) the most business cases for budget increases in application security spending today need to satisfy, at minimum, the following requirements:

1. Mitigation of risks of cyber threats targeting web application vulnerabilities
2. Mitigation of the impact of targeted attacks with new countermeasures
3. Meeting of industry specific security compliance requirements

Nevertheless, assuming the business cases can be made along these main requirements, CISOs today still have the difficult task to justify “how much” money should the company spend for application security and “where” to spend it. Regarding the how much, often it boils down to how much is needed to invest satisfy compliance requirements and pass the auditors check. When the focus is vulnerability risk management, the main goal is to fix high risk vulnerabilities and to reduce the residual risk to an acceptable value for the business. When the focus is to security incident management, the focus is how to quickly detect and respond to serious incidents.

Both for mitigating real risks (e.g. incidents, vulnerability exploits) and for mitigating non-compliance risks (e.g. unlawful non-compliance), the question for CISOs is "where" and "how" to prioritize the spending of the application security budget. Often the question is which countermeasure, application security process, activity, security tool yields “more bang for the money” for the organization. Regarding the "where" it comes down to balance correctly different application security and risk domains - to name the most important ones: business governance, security risk management, operational management that includes network security, identity management and access control and incident management. Since as a discipline application security encompasses all these domains, it is important to consider all of them and look at the application security investment from different perspectives.

Legal and Compliance Criteria for Application Security Budget Allocation

One of the main factors for funding an application security program is the need of the organization to comply with information security requirements mandated by the different technology security standards and regulatory bodies. Depending on the industry sector and the geographical location in which the organization operates, there will be several different types of information security standards and regulations that the organization need to comply with. Web applications that carry out payment transactions such as merchant type of ecommerce type of web applications that handle credit cardholder data are required to comply with the Payment Card Industry Data Security Standard PCI-DSS (Ref [4]). The requirements for the protection of cardholder data when is stored by the application includes (requirement # 3.4) rendering or encrypting the primary account number and masking (requirement #3.2) when it is displayed. The requirement for card authentication data such as PIN, CVC2/CVV2/CIDs is not to store these data at all even in encrypted form after a payment has been authorized. Credit cardholder data also need to be protected with encryption when is transmitted over open networks (requirement #4). These requirements for protection of cardholder personal account numbers and cardholder authentication data motivates the CISO to document internal security requirements to comply with these provisions and to adopt application security measures and assessment to verify that these requirements are met by the web applications that are in scope. Besides protection of cardholder data, PCI-DSS has provisions for the development and maintenance of secure systems and applications (requirement # 6), for testing security systems and processes (requirement # 11) and for the testing of web applications for common vulnerabilities (requirement #11.3.2) such as those defined in the OWASP Top Ten (Ref [5]). The need of compliance with the PCI-DSS requirements can be a reason to justify an additional investment in technology and services for security testing: examples include source code security reviews with SAST (Static Analysis Security Testing) assessment/tools and application security reviews with DAST (Dynamic Analysis Security Testing) assessments. For a merchant that develops and maintains web applications such as ecommerce web site that handle credit card payments, the main question is to whether allocate budget to application security measures and activities to comply with PCI DSS or to incur in fines (e.g. Up to $ 500,000 when credit cardholder data is lost or stolen. From this perspective, unlawful/noncompliance with a regulation/standard might be treated as another risk by the organization and as any other risk this could be mitigated, transferred or accepted. If risk of being non-compliant is accepted, CISO should considered that the data breach risk because of not implementing basic security controls such as data encryption but also input validation might be much higher than non-compliance. Consider for example the case of TJX Maxx incident. The company was non-compliant with PCI DSS when the breach of 94 Millions of credit card numbers were compromised, yet the costs for failing to encrypt or truncate card numbers as well as to identify and mitigate application vulnerabilities such as SQL injection were much higher (e.g. several hundredths of millions of dollars) than non-compliance costs (e.g. several hundredths of thousands of dollars). In the case of the U.S. banking sector, web applications that handle sensitive customer information and allow to process financial transactions such as to transfer money between different bank accounts (e.g. wires) are subjected to comply with strong authentication such as multi factor authentication (MFA) requirements in compliance with Federal Financial Institutions Examination Council (FFIEC) guidelines for online authentication (Ref [3]). From application security perspective this means that FFIEC requirements for authentication of online banking sites can justify budgeting for application security measures to secure design, implement and testing the provision of MFA controls in the web application. Web applications that store and process data that is considered personal data are required to protect it when such data is stored or processed in compliance with different privacy laws and regulations. For countries that are part of the European Union (EU), personal data is defined in EU directive 95/46/EC, for the purposes of the directive:[9] Article 2a: "personal data' shall mean any information relating to an identified or identifiable natural person ('data subject'); an identifiable person is one who can be identified, directly or indirectly, in particular by reference to an identification number or to one or more factors specific to his physical, physiological, mental, economic, cultural or social identity;" For most of the US States, protection of personal identifiable information (PII) is driven by data breach notification laws such as SB1386 where PII is more narrowly defined than in EU directive as as the individual's first name or first initial and last name in combination with any one or more of the following data elements, when either the name or the data elements are not encrypted: (1) Social security number. (2) Driver's license number or State Identification Card number. (3) Account number, credit or debit card number, in combination with any required security code, access code, or password that would permit access to an individual's financial account. (f) For purposes of these laws, "personal information" does not include publicly available information that is lawfully made available to the general public from federal, state, or local government records.

Web applications that process and store data that is considered personal private data by EU privacy laws or PII by US States data breach notification laws, need to implement security controls such as authentication, authorization, encryption, logging and auditing to protect the confidentiality, availability and integrity of this data. This information security requirement directly translates in application security requirements for building web applications that satisfy that requirement when storing and processing data that is either considered confidential or confidential PII. A request for budgeting an application security program for complying with these requirements is justifiable to the business in light of potential regulatory fines and legal costs that the organization might incur in case it is found unlawful and non-compliant with local privacy and data breach notification laws.

Risk Mitigation Criteria for Application Security Budget Allocation

From the perspective of the “how much needs to be spent in application security”, the executive management that will approve the budget need to know how much, of the overall security budget should be spent to reduce the likelihood of a similar security breach will not re-occur. For example, assume an online banking site has been breached and sensitive customer data have been compromised, the question is how much need to spent and which security measures should be targeted for spending. To answer these questions, a risk based approach should be used and some application specific security investment risk based criteria need to be adopted, some of these criteria are documented in this guide for CISO reference. These criteria can help CISOs to determine security costs in terms of either potential or occurred monetary losses due to accidents and attacks, and compare this with the cost of the investment in the security of applications. Nevertheless, any risk criteria can only be useful if based upon objective and not subjective considerations, such as using quantitative risk evaluations of the costs that the organization incurred because of security breaches exploiting web application vulnerabilities. Another important factor to consider by CISOs is exploiting the momentum being this either a negative or positive event. An increase in application security spending can be triggered by a negative event such a security incident (use survey again to confirm), since this shifts senior management's perception of risk. In this case the money is probably already being spent to remediate the incident and implement additional countermeasures. The main question then is what investment in application security will reduce the likelihood and impact of another similar incident happening in the future. One possible approach is to focus on web applications that might become a target for future attacks. A positive event might consist on a required technology upgrade of a web application such to introduce new functionality or migrate to a newer platform. This might also represent an opportunity to upgrade security technology and implement stronger security measures as well. Unfortunately, when a CISO need to decide where and how much money to spend in application security, the approach often followed is to use common sense. Often next year budget is based upon extrapolation of current year expenses and financing of new programs. These are all good criteria, a bad criteria is budgeting based upon the perception of risk among industry peers instead of an objective assessment of risks. The intent of this guide is to provide some objective assessment of risks and to approach application security investments from risk analysis and risk management perspective.

Application Security Budget Quantification

Risk quantification can be used as criteria for quantify how much money should be spent to mitigate the risk posed by insecure web applications. From risk analysis perspective, if a web application has been already attacked and sensitive data already being lost or compromised, it might already be brought to the attention of the business and in scope for identification, remediation and testing of the vulnerabilities that might had been the cause of the exploit/security incident. The next step, might be the design and implementation of risk prevention and detection security controls for similar web applications and software that might be at risk of similar attacks and incidents in the future. The main question for the CISO is which application security measures and activities should be targeted for spending to mitigate the risks of breaches of sensitive data due to malware and hacking attacks to web applications and software that is developed and managed by the organization. In Part II of this guide, we address how to target spending to mitigate the risk posed by specific attacks and vulnerability exploits. This helps in prioritizing risk mitigation by likelihood and business impact. From the perspective of risk management, application security spending matching all of the costs of the impact of a possible data breach is not justifiable. The main question for the CISOs is how much should be spent to mitigate the risk of data breach incidents, if not 100%, it is the 50%, 25% or 10% of all possible monetary losses? Also does the total loss estimate includes non-monetary losses such as reputation loss? In addition, if allocating a budget of 25% of the estimated potential losses due to data breaches is justifiable, how much of this 25% should be allocated in upgrading security in web applications that have a similar risk profile of the application being breached. Besides investing in security controls, how much should also be invested in improving secure software development/engineering processes, security training and improvements in application security testing? Part II and III of the guide will specifically address these questions and provide the rationale of which security controls and security activities to target for spending. For guiding the CISOs in making decisions on "how much money the organization need to budget for application security" we will focus on risk mitigation criteria rather than other factors such as percentage of the overall Information Technology (IT) budget and year over year budget allocation for applications security as fraction of overall information security budget that include compliance and operational-governance costs. A risk based application security budgeting criteria documented in this guide consist on the following:

1. Estimate of the impact of the costs incurred in the event of an security incident
2. Quantitative risk calculation of the annual cost for losses due to a security incident
3. Optimization of the security costs in relation to cost of incidents and cost of security measures
4. The return of security investment

We shall explain in the following sections of this guide each of these criteria and how can be used for quantifying how much money to spend in application security measures

Estimating The Impacts Of Security Incidents

There are two important factors to determine the risk of a security incident: these are the negative impact caused by the security incident and the likelihood (probability) of the incident. To obtain an estimate of the impact of the costs incurred in the event of a security incident, the key factor is the ability to ascertain the costs incurred due to the security incident. Examples of negative impacts to an organization because of a security incident might include:

1. reputation loss such as, in the case of publicly traded company, a drop in stock price as consequence of announced security breach (Ref[20]);
2. loss of revenue such as in the case of denial of service to a site that sells services or goods to customers (Ref[21]);
3. loss of data that is considered an asset for the company such as customer confidential data (Ref[22]), Personal Identifiable Information (PII) (Ref[23]), authentication data (Ref[24]) and trading secrets/intellectual property data (Ref[25])

In the case of a security incident that caused a loss of sensitive customer data such as personal identifiable information, debit and credit card data, the costs incurred by the organization that suffered the loss include several operational costs also referred as failure costs. In the case of a financial services company, these are the costs for changing account numbers, remission costs for issuance of new credit and debit cards, liability costs because of fraud committed by the fraudster using the stolen data such as for illicit payment transactions and withdrawal of money from ATMs. Often times, the determination of such “failure” costs is not directly quantifiable by an organization, such as when this monetary loss is not directly caused by a security incident, hence ought to be estimated as a possible impact. In this case, CISOs can use statistical data to determine the possible liability costs to the company in case of data loss incident. By using reported statistical data from data loss incidents, it is possible to estimate the costs incurred by companies to repair the damage caused by security incident that resulted in losses of customer sensitive data or identity loss. Appendix I-A contains a detailed discussion on the value of data and the impact of a security incident. The value of data will be different for each organization, but values in the range of $500 to $2,000 per record seem to be common.

     Data value: $500 to $2,000 per record

We will use this range for the remaining discussion, but each CISO needs to come up with some valuation of their own that can then be used to calculate the impact of a data loss.

One of the challenges of the calculation of the burden to the company because of a potential data loss is to get an accurate estimate of the amount of the loss x victim and of the probability or likelihood of such loss occurring. Statistical data about reported data loss incidents to breach notification letters sent to various jurisdictions in the United States collected by the Open Security Foundation's (OSF) DataLossDB (Ref [9]) show that the percentage of 2010 data loss incidents breaching a web interface is 9% and the percentage that reported as being an hack is 12%, fraud 10% and virus 2%. The highest reported incident by breach type is stolen laptop with 13% The data from OSF DataLossDB related to web as type of breach differ from the statistics of the Verizon’s 2011 data breach investigations report (Ref[1]) where hacking (e.g. brute force, credential guessing) and malware (e.g. backdoors, keyloggers/form grabbers, spyware) represent the majority of threats for security breaches (50% and 49% respectively) and attacks against web applications represent 22% of all attack vectors and 38% as percent of records being breached. These differences might be explained by the fact that the Verizon study is based upon a subset of data from the U.S Secret Service and does not includes for example cases related to theft and fraud that are instead counted on the overall OSF DataLoss DB statistical data. Furthermore, according to the Verizon report;" the scope of the survey was narrowed to only those involving confirmed organizational data breaches". In the case of OSF, survey data include data breaches covered by U.S State data breach notification law such as when resulting in disclosure of customer's Personal Identifiable Information (PII) and reported by organizations with notification letters sent to various jurisdictions in the United States.

Annualized Loss Of Security Incidents

In the cases when the impact of an occurred data breach due to a security incident is not being recorded and notified to the public in compliance with the data breach notification laws enforced by different countries and jurisdictions, it is necessary to estimate it based upon risk estimate calculations. Besides the calculation of liability costs based upon the value of data (refer to Appendix I-A: Value of Data), quantitative risk analysis can be used to estimate the spending for application security measures on the yearly basis such as by calculating the impact of a security incident on an annual basis. Quantitative risks can be calculated by the assessment of the Single Loss Expectancy (SLE) or probability of a loss as a result of a security incident and the Annual Rate of Occurrence (ARO) or the annual frequency of the security incident. By using quantitative risk analysis and using publicly available reports of data breaches, CISOs can estimate the amount that a given organization managing a web application would loose and therefore should spend on application security measures to mitigate the risk of a data loss due to the exploitation of an application vulnerability. The accuracy of this risk estimate depends on how reliable and pertinent the data breach incident is to application security. It is therefore important to choose the data carefully as this is being reported as being caused by an exploit of application vulnerabilities such as SQL injection (refer to Sony and TJX Max data breaches as good examples)

The SLE can be calculated with the following formula:

     SLE = AV x EF

Where, AV is the Asset Value (AV) and EF is the Exposure Factor (EF). The EF represents the percentage of the asset loss because of the realization of a threat or an incident. In the case of the 2003 US Federal Trade Commission (FTC) incident data this represents the amount of the population that suffered identity fraud and is 4.6%

Assuming the AV of 1 million accounts of $ 655,000,000 ( $655 per account based upon 2003 FTC data) and exposure factor of 4.6% (Ref [7]) the estimated SLE of the data breach incident is $ 30,130,000. Assuming a frequency of 1 attack every 5 years such as in the case of TJX Inc data breach incident (discovered in mid-December 2006 and due to SQL injection exploits) the ARO is 20% hence the estimated annual loss or Annual Loss Expectancy (ALE) can be calculated using the formula:

     ALE = ARO x SLO 

The calculated Annual Loss Expectancy (ALE) for data loss incident is therefore $ 6,026,000/year over 10 years.

Spend Optimization

Now the question is if, using quantitative risk analysis leads to an estimate to the optimal investment for application security measures. The honest answer is, not necessarily. The correct answer is to use cost vs. benefit analysis to determine the optimal value. By comparing the costs of security incidents against the cost of security measures it is possible to determine when this maximizes the benefit, that is, the overall security of the application. In case of software security costs for example, the cost due to software security failures including security incidents decrease as the company spends more money in security measures as shown in (FIG 1). The optimal investment in the security measures is the one that maximizes the security of the application and minimizes both the cost of security measures and of the cost incurred because of security incidents. According to an analytical study of costs vs. benefits of security (Ref [15]) the optimal investment is when the cost of security measures is approximately 37% of the estimated losses. For our example, assuming the total estimated losses of $ 6,026,000/year due to data loss incidents, the optimal expense for security measures, using this empirical value from the study, is $ 2,229,620/year.

Figure 1: Cost of Investment in software security measures against failure costs due to incidents that exploit software vulnerabilities. At the point (A) to the costs due to software security failures exceed of several order of magnitude the expenditure in countermeasures and the assurance on the security of the software is very low, on the contrary in (B) the costs of security measures outweigh the costs due to the software failures, the software can be considered very secure but too much money is spent for software security assurance. In point (C) the cost of losses is nearly two times larger costs of security measures while in point (D) the costs due to incidents is equal to the cost of the security measures. The optimal value for spending of security measures is the one that minimizes both the cost of incidents and security measures and maximizes the benefit or the security of the software.

Return on Security Investment

Finally, it is important to determine the most efficient way to spend the application security budget from a perspective of this being an investment. If the CISO considers application security spending as an investment rather than an expense, for example, the budget can be justifiable as additional savings the company gets because of the investment in security. The factor to calculate the savings in terms of investment in security is the Return on Security Investment (ROSI). The ROSI can also help to determine if the investment in countermeasures to thwart hacking and malware attacks is justifiable: if the ROSI is not positive, the investment is not justifiable while if it is null, it does not yield any savings or investment returns. There are several empirical formulas to calculate ROSI; one is to factor of the savings for the data losses avoided over the total cost of the countermeasures. Assuming the Total Cost of Ownership (TCO) (Ref [16]) for the cost of countermeasure is $ 2,229,620 (previously calculated as optimal value of expense in countermeasures x year) that include development costs and acquisition of the new technologies, processes, tools as well as operating and maintenance costs it is possible to calculate the savings. ROSI can be calculated using the following empirical formula (Ref [17]):

      ROSI= [(ALE x % of effective risk mitigation) - cost of countermeasures]/cost of countermeasures

With this ROSI formula, assuming that the ALE is $ 6,026,000 and that the effectiveness of the mitigation is 75% (assume for example, in the case of a SQL injection, risk mitigation as defense in depth such as different layer of mitigation that include use of prepared statements/stored procedures in source code as well as filtering of malicious characters at the web server and application server), the cost of countermeasure is $ 2,229,620, the ROSI to the company is 102% per year. Since there is return of investment, the spending in countermeasures is worth it and will make the company save money. The best use of ROSI (Ref[17]) is to compare alternative investments in security countermeasures such as to decide whether to invest in the development of a new countermeasure or extending the capabilities of an existing one.

As a comparative measurement for example, ROSI can be used by CISOs to determine which software security is more efficient or yields the organization the higher savings and returns on the investment. According to research of Soo Hoo (IBM) (Ref [18]) on ROSI of the various activities of software security in software development cycle, the maximum return of investment (21%) or a savings of $ 210,000 on an investment of a $ 1 Million Secure Software Development Lifecycle (S-SDLC) program for example is obtained when the investment is in activities that aim to identify and remedy security defects during the design phase such as threat modeling. The return of investment is 15% when the defects are identified and remedied during implementation (code) such as with source code analysis and of only 12% when these are identified and remedied during the testing-validation phase such as with ethical hacking/pen tests. The best investment in application security is therefore in activities that aim to identify defects as early as during the design phase of the SDLC, in essence, the more CISOs think about investing in software security engineering programs especially threat modeling/architectural risk analysis, the more they'll save on the costs of implementing and fixing security issues later in the SDLC such as during the validation phase or when the application is already in production in response to security incidents.

Conclusion

Finally, it is important to notice that the criteria dealt with herein are based upon empirical formulas and are as good as the data used. The more accurate is the data the more accurate are the risk and cost estimates. Nevertheless, when these risk-cost criteria are used consistently and based upon quantitative risk decisions and objective security cost considerations, can be used by information risk management decision makers such as CISOs to decide if the investment in application security is financially justifiable from risk and business impact perspective. Since investment in security has to be justified in business terms, these risk-cost criteria can be used for business cases as well as to decide how much to spend and where to spend in application security measures.

Part II: Selection of Application Security Issues to Target Spending

Introduction

Once a web application has been targeted by an attack and the organization has suffered either a data breach incident or fraud as result of it, it is important to understand the root causes (e.g. vulnerabilities, control gaps) of the incident and to invest in security measures that will prevent such incident to occur again. In this section of the guide, we address how to target spending to mitigate the risk posed by specific attacks and vulnerability exploits that caused data breach incidents. As best practice, we are not advocating to fix only vulnerabilities that might have been the cause of the incident even if these are the ones that need to be prioritized first for remediation to limit further damage. Vulnerabilities that might have been already exploited to attack the web application certainly represent the highest probability to be also exploited in future targeted attacks. The main question for the CISO is also to whether the same vulnerabilities can be used in attacks in the future against web applications that have a similar functionality and type of data. Nevertheless, the application might have other type of vulnerabilities that might be opportunistically exploited by an attacker. These are vulnerabilities that either enable or facilitate an attacker to conduct the attacks against web applications. The main point to is that since the risk of data breaches and online fraud are a factor of likelihood and impact of vulnerabilities, it is important to consider likelihood and impact as factors to determine which issues to target for spending. In general, vulnerabilities are prioritized based upon technical risks not business impact, for example, vulnerabilities that yield high technical risks are prioritized for remediation over low risk ones. A vulnerability of high technical risk can be SQL injection for example independently from the data asset and the value that such asset has for the organization. Clearly if that SQL injection vulnerability is affecting either authentication or confidential data might represent a very different risk to the organization than a SQL injection vulnerability that might affect data that is considered of low risk for the organization such as marketing research data for example. The impact might be more of reputation risk in this case rather than data breach risk. In part I of this guide we provide business cases that CISO can use to request budget for application security. Application security budget typically need to cover several information security and risk governance needs. Besides the usual need to spend for compliance with information security standards, policies and regulations, CISO might advocate additional budget to address mitigation of increased risks of data breach incidents. One critical factor is to quantify the impact of the data breach incident that already occurred. This implies that the CISOs are authorized to access data in relation to data breach incident such as incident reports filed by the Security Incident Response Teams (SIRT), data from legal in relation to law suits and regulatory fines and fraud data that includes amount of money losses incurred because of online fraud. All this type of information is essential to determine the overall impact. In absence of this data, the best the CISO can do is to use data breach incident data from public sources and data breach incident reports. In part I of this guide, we provided some examples of how this data can be used to estimate impact. We documented what are the critical factors to estimate impacts of data breaches: these as the value of the data assets (e.g. Customer confidential and personal identifiable information, credit cards and bank account data) and the liability for the organization in case these asset are lost. Once the potential business impact of a data breach is estimated, the next step is to determine how much should be spend to mitigate the risk. At high level, this is a risk strategy decision that depends by the organization risk culture and the organization priorities for mitigating risks. Depending on the type of the organization, the number one priority can be "to not to be caught in unlawful non-compliance" such as in case of suffering a data breach and additionally failing to comply with compliance with PCI-DSS standards. This can be the case of small company that provides online payment processing services and who could lose business from credit card issuers and additional fines, law suits and audit and legal costs. For an organization such as an engineering or research organization whose patents and trading secrets are a critical assets, the protection from internal threats of commercial or country sponsored spying might represent number one priority. In general, it is important to address to application security as a business enabler for protecting digital assets whose value is represented in terms of costs of security measures vs. benefits in protecting the digital assets. In part I of the guide we present one criteria that can be used is the one that optimize spending by maximize risk mitigation value while minimize the security costs. Another criteria, is to consider security not as a tax but as an investment, this criteria is the Return of Investment in Security (ROSI). The ROSI can be used for making both tactical and strategic risk mitigation decisions. Tactically, ROSI can be used to decide which security measures should be targeted for spending by considering the cost vs the effective of the measure in mitigating the impact of the data loss. Strategically, ROSI can be used to decide which application security activities to invest in the SDLC such as the ones that will bring money savings in the long term.

Targeting Vulnerabilities with the Most Impact

In this part of the guide, we would like to address a more tactical approach that helps CISOs to decide where to spend the budget in application security by addressing first the immediate needs such fixing security issues that were exploited in a data breach security incident and vulnerabilities that have been exploited in publicly reported incidents affecting similar organizations and web applications. To estimate the probability that an attack will cause a data loss, we would need to identify sources of attacks that correlate data from different type of publicly disclosed incidents (e.g. data loss, denial of service, defacement etc) with sources of monitored attacks seeking to exploit specific web application vulnerabilities. To estimate the probability of a specific web application vulnerability exploit, we can refer to data reports from the Web Hacking Incident Database (WHID) (Ref [10]). The WHID is a Web Application Security Consortium (WASC) project to provide statistical analysis information of web application security incidents collected from public sources. in 2010 WHID categorized 222 incidents and observed that 33% of the incidents aimed to take down web sites (e.g. with Denial of Service), 15% aimed to deface web sites and 13% to steal information. Among the overall type of attacks the ones that sought to exploit application vulnerabilities such as SQL injection were 21%. By using 2010 WHID data of reported incident and analysis, the overall probability of an attack aimed to steal information by exploiting of a SQL injection vulnerability is therefore 13 % x 21 % = 2.7%. Since SQL injection was also reported to be used for defacement, this ought to be considered as rough estimate. In another survey of malicious web attack traffic observed over a period of six months, December 2010 through May 2011 from the security company Imperva (Ref [11]), SQL injection was identified in 23% of the attacks as third most prevalent after cross site scripting, the second most prevalent in 36% of the attacks and directory traversal as the most prevalent in 37% of all the attacks. By comparing WHID and Imperva web attack surveys, an order of magnitude of 21-23% for attacks exploiting SQL injection vulnerability seems an acceptable rough estimate. By assuming the cost of data loss of security incident for a financial organization of $355/record (Ponemon Institute 2010 data), and that the probability that such incident exploits a SQL injection vulnerability is 2.7% (WHID 2010 data), the 2010 liability for a company's web site such as online banking for a data loss of 1 million records is thus $ 9,585,000. With this figures a 2010 budget of $9 Million spent by a financial organization for application security measures specifically focused to prevent risks of data losses due to SQL injection attacks would have been justifiable.

Assuming that I will spend as much in security measures, this is the maximum amount estimated for expenses in security measures to thwart SQL injection attacks that includes acquisition of technology for secure software development, documentation, standards, processes, tools as well costs for the recruitment of qualified personnel and secure coding training especially for web developers. Normally this dollar figure ought to be considered a maximum value since assumes for example a total loss of the customer data. It is important to notice that injection vulnerabilities are considered by OWASP (Ref [5]) (2010 A1-Injection) the most critical web application security risks for opportunistic vulnerability exploits. OWASP rates the risk of data injection, including SQL injection vulnerability, as severe since "can result in data loss or corruption, lack of accountability, or denial of access and sometimes lead to complete host takeover". The business impact that we calculated as liability for a medium size financial services company (1 million registered online banking users) assumes that the value of the data assets can be stolen by a threat agent to cause tangible harm to the company. Historically, SQL injection attacks have been of high impact and in the United States, have been associated with the largest data breach incidents ever committed and prosecuted. In the August 2009 U.S. indictment case (Ref[12]) against Albert Gonzalez (also indicted in May 2009 in Massachusetts for the TJX Inc breach) and other two Russian hackers, SQL injection attacks were used to break into 7-Eleven network in August 2007 resulting in the theft of credit card data. Allegedly, the same kind of attack was also used to infiltrate Hannaford Brothers in November 2007 which resulted in 4.2 million debit and credit card numbers being stolen and to steal 130 million credit card numbers from Heartland Payment Systems on December 2007. In 2010, Albert Gonzalez was found guilty and sentenced to serve 20 years in federal prison while Heartland paid about $ 140 million in fines and settlements because of the security breach.

Targeting Threats to Web Applications with New Countermeasures

The goal of this chapter is to provide CISO with selection criteria for countermeasures that can be used for proactive mitigation of threats against web applications. The choice of which countermeasures are critical to mitigate threats need to start first with the understanding of the threat scenario and the threat agents especially of hacking and malware and how this threat scenario adversely affects security controls in web application to compromise the data assets as well as the critical business functions. One critical tool in the hands of CISO is the use of risk frameworks that capture the risk profile of the application as a target for hacking and malware as well as the analysis of existing web application control gaps. The selection of countermeasures needs to follow criteria such as defense in depth and need to consider both the costs and the effectiveness of new countermeasures in mitigating the risks.

Targeting the Risks of New Technologies Impacting Web Applications

The goal of this chapter is to guide the CISO on the consideration of the security risks posed to the organization by the adoption of new technologies. The term “technologies” is used herein to include recent examples of technologies that impact web applications such as mobile technologies, web 2.0 technologies, and cloud computing Software as a Service (SaaS). As technologies evolve, it is important for the CISO to understand the security risks introduced by the adoption of these new technologies since these might represent new opportunities for attackers to attack both web applications and the data. The increased risk to web applications due to the adoption of new technologies includes the increased exposure/attack surface such as in the case extending web applications to mobile devices, the introduction of new class of client and server side vulnerabilities such as in the case of Web 2.0 and the increased risk of loss of data and transaction integrity due to the use of cloud computing. In order to target the mitigation of the risks due to the adoption of these technologies, CISO need to have a clear picture of the risks that are introduced and decide to invest in new type of application security assessments, tools and security measures to mitigate the risks.

Part III: Selection of Application Security Processes to Target For Spending

Introduction

Mitigating the risk of attacks that seek to exploit of web application vulnerabilities as well as potential gaps in protective and detective controls is one of the CISOs main concerns. In the case when vulnerability is only found after a security incident occurred, the next step is to fix the vulnerability and limit and further impact. Typically this involves retesting of the vulnerability and make sure is being fixed so ca no longer be exploited in the future. If the incident is due to a gap of a security control such as for example a failure to filter malicious input or to detect the attack event, the next logical step is to implement a countermeasure to mitigate the risk. To makes such decisions, the CISO need to consider both the risks of vulnerabilities as well as the weaknesses of security control measures to make a decision on how to mitigate the risks. Typically fixing a vulnerability involves a vulnerability management cycle that includes identifying the vulnerability, fixing it and then re-testing it to determine that is no longer present. In case of countermeasures the test that the countermeasure is effective in preventing and detecting an attack vector can also be tested with a functional security test after countermeasure is deployed. The decision to which countermeasure to deploy might depend on different factors such as the cost of the countermeasure vs. the business impact of the incident as well as on how risk mitigation effective is the countermeasure by comparing with others. The next step for the CISO after the security incident is under control is to make sure any vulnerabilities are fixed and countermeasures are deployed to mitigate the risk. In this section of the guide we focus on application security measures that are most cost effective to target the issues identified in Part 2. For example how to divide budgets across software security activities such as secure code training, secure code reviews, secure verification/test and issue and risk management". The results of the CISO survey will be used to determine where money is spent and in which activities to determine if application security money is spent effectively (e.g. focus on security build in activities vs security bolt-on security activities). The results of the CISO survey will also be used to identify if standard application security controls are used to highlight which OWASP objects are most useful (e.g. SAMM, ESAPI, CLASP, Security Development, Coding and Testing Guides) "

Targeting Specific CISO Roles & Responsibilities

The adoption of application and software security processes within any given organization varies greatly depending on the type of the organization’s industry, the size and the different roles that CISO play in the organization. The source of application security budget also varies depending on the size and the type of the organization. For CISOs of large organizations, typically the budget for application security is part of the overall budget allocated by information security and operational risk departments. For these CISOs, one the main reasons for the adoption of new application security activities, guides and tools such as the ones that OWASP provides, is first and for the most to satisfy compliance and to reduce risks to the organization’s assets such as web applications and software. Compliance varies greatly depending on the type of industry and clients served by the organization. For example, organizations that produces software that implements cryptography for use by governments such as the department and agencies of the United States Federal government need to comply with Federal Information Processing Standards (FIPS) 140. Organizations that produce software and applications that handle cardholder data such credit and debit card data for payments need to comply with is the Payment Card Industry Data Security Standard (PCI DSS). CISOs of small organizations typically have responsibility on both security and information technology functions that might also include the compliance of applications and software with technology security standards such as FIPS 140 and PCI-DSS. Compliance with security technology standards represent an opportunity for promoting secure development and testing within the organization such as by using OWASP security testing guides for achieving security certifications for applications and software products. Compliance with PCI-DSS requirements for example might already require the organization to test web applications for a minimum set of common vulnerabilities such as the OWASP Top 10. The budget allocated by the IT department for achieve certifications with technology security standards such as FIPS-140 and PCI-DSS can also be used for promoting secure coding guides such as the OWASP secure coding guide and invest on static code analysis tools. For example, in the case of compliance with PCI-DSS, CISOs might opt for static code analysis to satisfy the requirement 6.6 of PCI-DSS. CISOs of small organizations can also use defect management metrics to make the business case in which phases of the SDLC to invest in security and improve both software quality as well as security. For example, since most of the quality and security bugs are due coding errors, it is important for CISOs to emphasize to the IT department the need of secure coding processes, standards and training for developers since focusing on these software security activities also leads to cost savings for the organization. A study from NIST about the cost of fixing security issues for example has shown that the cost of fixing a coding issue in production is six times more expensive than fixing it during coding. To achieve these money saving and efficiency goals, CISOs can work together with the engineering department managers to promote application and secure software initiatives. For CISOs of large organizations, whose focus is information security and risk management, one of the main requirements besides compliance is to introduce efficiencies and save the money spent for existing security processes, including, application security. Since the information security department allocates budgeting, any request for budget of application security need to be justified by improving security and by reducing risks. Security and risk reduction goals are aligned by improving security test processes with use of better tools and training for developers. For CISOs of large organizations, promoting a software security initiative is also justified by the return of investment in the overall application security program and processes, specifically as reduction in the cost of fixing vulnerabilities because of developers following secure coding standards, conducting secure code reviews and security teams conducting security testing for vulnerabilities earlier than the validation phase of the SDLC. OWASP provides secure development guides and secure coding guides as well as training modules that can be used to achieve this cost saving goals. Often CISOs need to justify the budget for application security by taking into consideration the different needs of security and business departments. For CISOs that serve in financial organizations for example, security is often a compromise with security and business goals. In this case, it is important for CISOs to be able to align application security programs with the business goals and when these goals not align, to focus on the ones that do. For example, by focusing on improving both software quality and security and by reaching a compromise in the case security impacts negatively the customer experience so different security options need to be considered. In the case the business is sponsoring a new application development project, CISOs can use this as an opportunity to promote new application security features for the application and work together with project managers by achieving compliance with security standards, improving security by design and by coding and yet achieving overall cost savings for the overall project. For CISOs of both large and small organizations, application vulnerability metrics plays an important factor in making business cases for budgeting application security. Security metrics such as vulnerabilities found on the same applications over a certain period of time when application security activities have been rolled out for example, can demonstrate to senior managers and company executives that the adoption of application security processes, training and tools ultimately helps the organization to deliver web applications and software products that have a fewer number of vulnerabilities and pose less risk to the organization as well as the customers.

Targeting Software Security Activities and S-SDLC Processes

Since a large number of vulnerabilities in web applications are caused by insecure coding, it is important that the CISO recognizes the importance that secure software has in improving the security of the web application. The causes of insecure software might depends by different factors such as coding errors, not following secure coding standards and security requirements, integration with vulnerable software libraries, missing secure code review processes and security testing and formal secure code training and awareness for software developers. From CISO perspective, it is important to understand that software security is a complex discipline and requires a special focus in security processes, tools as well as people skills. It is also important to recognize that investing in software security helps the organization to save money spent in web application vulnerability remediation costs in the future. By investing in software security initiatives, organizations can focus on fixing vulnerabilities as early as during coding phase of the Software Development Life-Cycle (SDLC) where is cheaper to identify, test and fix them than during the validation phase. Today, also thanks to OWASP, software security has matured and evolved as a discipline. For example, several organizations already adopt software security best practices within their software development processes such as the documentation of security requirements, following of secure coding standards and use of software security testing tools such as static source code analysis tools to identify vulnerabilities in source code before releasing source code to be build and integrated for final integrated and user acceptance tests. By integrating software security activities in the SDLC, organizations can produce software and applications with a fewer number of vulnerabilities and lower risks than software and applications that don’t. The question for CISO is rather not IF but WHICH and HOW software security activities can be integrated as part of the SDLC. According to the National Institute of Standards and Technology’s Special Publication 800-30, “Effective risk management must be totally integrated into the SDLC ... [which] has five phases: initiation, development or acquisition, implementation, operation or maintenance and disposal.” The integration of security in the SDLC process begins by identifying the information assets that the software will be processing and by specifying requirements for confidentiality, integrity, availability and auditability. The next step consists on the determining the value of information assets, identifying the potential threats and determine the requirements for application security controls such as authentication, authorization and encryption to protect the confidentiality, integrity and availability of the data assets. A comprehensive set of security requirements need to also include requirements to implement secure software by following certain security and technology standards, security approved technologies and platforms as well as security checks prior of software integration with other vendors software components/libraries. When software is acquired as either part of the commercial off-the-shelf (COTS) or as free open source (FOSS) for example, it is important for CISO to have a process in place to validate this type of software libraries against specific security requirements prior to acquiring them. This could provide the CISO of the organization a certain level of assurance that the acquired software is secure and can be integrated with the application. In that regard, OWASP had developed a legal project and a contract annex of a sample contract that included security requirements for the life cycle so that COTS products would be more secure. In cases when the CISO of the organization has also responsibility over promoting a software security process within the organization, it is important not to take this goal lightly since usually requires careful planning of resources and development of new processes and activities. Fortunately today, several “Security in the SDLC” (S-SDLC) methodologies can be adopted by CISOs to incorporate security in the SDLC. The most popular S-SDLC methodologies used today are Cigital’s Touch Points, Microsoft SDL and OWASP CLASP. At high level, these S-SDLC methodologies are very similar and consist on integrating security activities such as security requirements, secure architecture review, architecture risk analysis/threat modeling, static analysis/review of source code, security/penetration testing activities within the existing SDLCs used by the organization. The challenge for the integration of security in the SDLC from CISO perspective is to make sure that these software security activities are aligned with the software engineering processes used by the organization. This means for example to integrate with different types of S-SDLC s such as Agile, RUP, Waterfall as these might be already followed by different software development teams within the organization. An example on how these can be integrated within a waterfall SDLC as well as iterated within different iterations of a SDLC process is shown herein (TBD figure).

From CISO perspective, adopting a holistic approach toward application and software security leads to better results since can align with information security and risk management already adopted by the organization.. From information security perspective, the holistic approach toward application security should include for example security training for software developers as well as security officers and managers, integration with information security and risk management, alignment with information security policies and technology standards and leveraging of information security tools and technologies used by the organization. Besides of following a holistic approach toward application security that considers other domains it is also important for the CISO to consider what the organization capabilities are from day one in building software security and plan on how to integrate new activities in the future. Measuring the organizations capabilities in software security is possible today with software security maturity models such as the Build Security In Maturity Model (BSIMM) and the Software Assurance Maturity Model (SAMM). These models can also help the CISO in the assessment, planning and implementation of a software security initiative for the organization. These maturity models are explicitly designed for software security assurance. These models, even if are based upon empirical measurements, are feed from real data (e.g. software security surveys) hence allow to measure organizations against peers that already had implemented software security initiatives. By allowing their organization’s secure software development software practices to be measured using these models, CISOs can compare their organization secure software development capabilities against other software development organizations to determine in which software security activities the organization either leads or lags. For the software security activities for which the organization is lagging, BSIMM and SAMM measurements allow the CISO to construct a plan for software security activities to close these gaps in the future. It is important to notice that these models are not prescriptive that is, are not telling organizations what to do but rather to measure security activities in comparison with similar organizations in the field. The models are organized along similar domains, governance, intelligence, SSDL touch points, deployment for BSIMM and governance, construction, verification, deployment for SAMM. SAMM measurements are done in three best practices and three levels of maturity for each domain. BSIMM measurements cover 12 best practices and 110 (tbc) software security activities. The maturity levels help the CISO to plan for the organizational improvements in software security processes. Software security improvements can be measured by assigning goals and objectives to reach for each activity. For CISOs that either have already started to deploy a software security initiative such as S-SDLC within their organization or that just plan it in the future, the measurements that a model such as BSIMM and SAMM provide are important measurement yard sticks to determine in which application security activities to focus spending. If not already familiar with BSIMM and SAMM, CISOs can also refer to the Capability Maturity Model (CMM) and the various maturity levels to plan for the organization secure software development process capabilities. Like BSIMM and SAMM, CMM is also an empirical model whose goal is improve the predictability, effectiveness, and control of an organization's software processes. In CMM for example, these are five levels that can be used to measure how the organization moves up to different levels of maturity of software engineering process: initial, repeatable, defined, managed, optimizing. In the first level (initial), the software engineering process is ad-hoc and used by the organization in uncontrolled and reactive manner. As the software development organization reaches level 2, the software development processes are repeatable and is possible to provide consistent results. When an organization reaches level 3, it means that it has adopted a set of defined and documented standard software development processes and these are followed consistently across the organization. At a level 4, that is managed, a software development organization has adopted metrics and measurements so that software development can be managed and controlled. When a software development organization is at level 5, optimized, the focus is on continually improving process performance through both incremental and innovative technological change and improvements in software development. For CISOs that are more familiar with CMM levels, it is possible to map software security from the initial (level 1) to optimized (level 5) via repeatable (level 2), defined (level 3) and managed (level 4) levels of software security assurance. An example to chart the progress toward the obtainment of these levels of maturity for software security is the case of a CISO that had already adopted web application penetration testing and would like to start to introduce an new process for software security such as secure code reviews/static code analysis. Organizations that start at CMM Level 1 (Initial) do not have a process for software security yet and just follow a catch and patch approach. Software security issues are identified by an occasional ad-hoc static code analysis testing of the application software. At this level, the organization maturity in software security practice consists on running a static source code analysis tool in reaction of other events such as to validate a vulnerability previously identified in a web application penetration test. At CMM Level 2, the organization has already adopted a secure code review process and use static analysis tools. This process produces repeatable results such as when is source code is tested by the same security testers within the same team that use the same process and tools. At this level, the secure code testing process can be repeated to produce consistent results (e.g. security issues) but is either not adopted across all software development groups within the same organization or these are still different. The organization reaches the CMM Level 3 when has adopted a secure coding review/static source code analysis processes that is the same across the different security teams within the same organization. The process is also defined by standard documentation so every security team follows the same standards and the results are consistent across different teams within the same organization. Organizations at level CMM 4 (managed) allow the CISO to manage the software security issues and measure and manage the risks of insecure software. Organizations at level CMM 5 (optimized) allow the CISO to optimize the software security development processes for increased return of security investment, produce security cost savings and improve risk mitigation/reduction.

How to choose the right tools from OWASP and other organizations

Depending on the overall security level and risk profile of the organization unit different tools and standards can be particularly useful for the CISO in advancing his security strategy. Note, following the risk discussions in the previous chapter, depending on the risk profile of different business units, the security strategy can actually be different based on their different individual risk scenarios and different regulatory requirements. For example a financial department may require a substantially stronger security posture, while an internal web page announcing the lunch menus of the cantine may be sufficiently protected with basic security measures (though to the authors knowledge in military settings, even the lunch menu can be considered as confidential information as further information about supply logistics etc. could be derived from that). Based on these different risk profiles different tools and standards may be more relevant for the project and organizational unit in question. In general tools can be classified in various categories (and so are also the OWASP projects):

Maturity

- Stable (a project or tool that is mature and constantly maintained to a good quality) - Beta (relatively proven, though not to optimal quality) - Alpha (this usually reflects a good first prototype, but still a lot of functionality may be missing or not up to standard) - Inactive (former projects that have been retired or deprecated or that at some point have been abandoned). Obviously for a CISO, the most interesting projects and tools would be stable and reliable ones. He can rely on a certain proven quality, and on them being available and maintained to a certain degree in the future days to come. Beta projects can also be very valuable, as they may represent projects that have not finished their full review cycle yet but are already available for early adopters and can help to build good foundations for your security programs and tools going forward. A second dimension would be the various:

Categories

Usually OWASP projects are divided in either Tools or Documentation. And by the category of use: Protect, Detect and Life Cycle. These categories can help the manager to quickly navigate the large portfolio of OWASP tools available and more easily find the right project for his current needs. Please find a page of the various OWASP projects classified by categories here.

People, Processes and Technology

The CISO can also choose to achieve his security goals through three main ways. People, Processes and Technology. Managing the organization it is usually important to shape all three pillars to achieve the best impact throughout your organization. Focusing on only one or two of them can leave the organization vulnerable.

People

This will address the training and motivation of staff, suppliers, clients and partners. If they are well educated and motivated, the chances of malicious behavior or accidental mistakes can dramatically be reduced and many basic security threats can be avoided.

Processes

If an organization becomes more mature, the processes will be well defined and in fact channel and enable the work force to do things the "right way". Processes can ensure that the actions of the organization became reliable and repeatable. For example with well-defined standard operating procedures, the incident response process will be reliable and not rely on ad hoc decisions that would before have varied with the individual decision maker. In highly mature organizations, the business and IT processes will be constantly evaluated and improved. If a failure happens, improved processes can allow an organization as a whole to learn from past mistakes and improve its operation to more efficient and secure ways.

Technology

In general technology can guide and support people by providing good training and knowledge, by being engaging and motivating to work with. And Technology can enable an organization to make the following the right processes easy by providing good tools, while making it hard to deviate from the right path for malicious users. For example good technology would automate access controls and authentication and make them very simple for the authorized user, while denying access or privileges to an unauthorized attacker. And last but not least a number of automated tools can in the background help and support the people and organization in their work to defend against risks more effectively and more efficiently. Many of the security standards and tools (in OWASP and other bodies) can also be seen as focusing on parts of this framework. For example, staff training will enable the people to build their security understanding and do the right thing, while the various SDLC models can help an organization establish the right level of processes for its development and incident response mechanisms.

Benchmarking & Maturity

One of the very first steps for a CISO is to understand his current situation by reviewing the current security maturity of his organization or the individual department and benchmark it against peers or his target security posture. There are several maturity models published, with variations in focus and depth of detail. Usually they share a number of good practices and for a CISO it may be advisable to review whether his organization is elsewhere already using one of the maturities models and possibly align with this for an easier initial benchmark of his organization. In the medium term the decision for which maturity model to use, would be driven be question of: what level of detail is required, are we required by a regulatory body to use and report based on a specific type of maturity model, can our model be easily integrated with the organization’s culture and common reporting information, ... In the end, the author believes that most maturity models can equally fulfill your basic needs and that it will be up to the tactical judgment of the CISO to decide which model to use. The openSAMM maturity model has in the past been developed by OWASP and is a very mature ("stable") project, that offers a fairly lightweight way in analyzing your current security maturity and benchmarking your organization against your peers and your targets. See also: Software Assurance Maturity Models (SAMM), OWASP, http://www.opensamm.org/ Other maturity models can be taken from BSIMM, CMM (Common Maturity Model), the ISO-2700x series

Part IV: Metrics For Managing Risks & Application Security Spending

Introduction

The aim of this part of the guide is to help CISO to manage the several aspects of the application security program specifically risk and compliance as well as application security resources such as processes, people and tools. One of the goals of the application security metrics is to measure application security risks as well as compliance with application security requirements mandated by information security standards. Among critical application security processes that the CISO need to report of and manage are web application vulnerability management. It is often CISO responsibility for example to report the status of the application security activities to senior management such as the status of application security testing and software security activities in the SDLC. From the risk management perspective it is important that the application security metrics allow to reports on the technical risks such as the un-mitigated vulnerabilities for the applications that are developed and managed by the organization. Another important aspect of the application security metrics is to measure coverage such as the percentage of the application’s portfolio regularly assessed in application security verification program, the percentage of internal apps vs. external apps covered, the inherent risk of these apps and the type of security assessments performed on these applications and when in the SDLC are performed. This type of metrics helps the CISO in reporting on application security process compliance and application security risks to the head of the information security as well as to the application business owners. Since one of the CISOs responsibilities is to manage both information security and applications security risks and to make decisions on how to mitigate them, it is important for this metrics to be able to measure these risks in terms of vulnerability exposure to the organization’s assets that include application’s data and functions.

Application Security Process Metrics

The goal of the application security process metrics is to determine how good are the organizations application security processes in meeting the security requirements set forth by application security policies and technical standards. For example an application vulnerability process might include requirements to execute vulnerability assessments on internet facing applications every six or twelve months depending on the inherent risk rating of the web application. Another vulnerability process requirement would be to execute security in the SDLC type of processes such as architecture risk analysis/threat modeling, static source code analysis/secure code reviews and risk based security testing on web applications that store customer’s confidential information and whose business functionality is a critical service to customers. From the perspective of process coverage, one of the goals of this metrics might be to report on the coverage of application security process such as to measure how web applications fall in scope for application security assessments to identify potential vulnerability assessment gaps based upon application type and the application security requirements. This type of metrics helps the CISO to provide visibility on process coverage as well as the status of the operational execution of the application security programs. For example the metrics might show (e.g. in red status) that some of the application security processes in the SDLC such as secure code reviews are not executed in some of the high risk rated applications and flag this as an out of compliance issue with the security testing requirements. This type of metrics allow the CISO to prioritize resources by allocating them on where is most needed to comply with the standard process requirements. Another important measurement for application security testing is to measure the time of when the application security processes are scheduled and executed to identify potential delays in the scheduling and execution of application security processes such as secure code review/static source code analysis as well as ethical hacking/web application pen testing.

Application Security Risk Metrics

One of CISO responsibilities is to manage application security risks. From technical risk perspective, application security risks might be due to vulnerabilities in the applications that might expose the application assets such as the data and the application critical functions to potential attacks seeking to compromise the data and/or the critical functions that the application provides. Typically, technical risk management consists on mitigate the risks posed by vulnerabilities by applying fixes and countermeasures. The mitigation of the risk of these vulnerabilities is typically prioritized based upon the qualitative measurement of risks. For example, for each web application that is developed and managed by the organization that would be a certain number of vulnerabilities identified at high, medium and low risk severity. The higher the number of high and medium risk vulnerabilities the higher is the risk to the application. The higher the value of the data assets protected by the application and the criticality of the functions supported, the higher the impact of these vulnerabilities on the application assets. One important emphasis that is given in the vulnerability metrics is the determination of the number of vulnerabilities that are still not fixed. A given number of application vulnerabilities might still be “open” that is not yet fixed in production environment: these represent a risk to the organization and require the CISO to prioritize the risk mitigating action such as “closing” the vulnerability within the compliance timeframes that is deemed acceptable by the application vulnerability management standards. Another CISO important metrics for the managing of information security risks is the reporting of security incidents for web applications that are developed and/or managed by the organization. The CISO might gather this data from reported from SIRT (Security Incident Response Team Incidents) that affect a given web application such as breaches of data as results of an exploit of a vulnerability. The correlation of the security incidents reported for a given web application with the vulnerabilities reported by security testing allows the CISO to prioritize the risk mitigation effort on mitigating vulnerabilities that might cause the most impact to the organization. Obviously, waiting for a security incident to occur to decide which vulnerabilities to mitigate is symptomatic of a reactive rather than proactive approach toward risk management. Risk proactive organizations do not wait for security incidents to occur but rather learn from information about attacks and threat intelligence and use that information to take proactive risk mitigation measures such as to develop and implement countermeasures yet mitigating all high risk known vulnerabilities that might be potentially exploited in an incident to cause the most impact to the organization. The CISO can use threat intelligence reports as well as metrics from monitored application layer security events such as from SIEM (Security Incident Event Management) systems to assess the level of risk. Unfortunately today, most of security incidents are discovered and reported only after months from the initial intrusion or data compromise. A security metrics that is actionable toward preventing risks of attacks is of critical importance for CISO since it might allow deciding which web applications to put under alert and monitoring and to act quickly in the case of an attack. For example a threat alert of a possible distributed denial of service against online banking applications might allow the CISO to put the organization on alert and prepare to roll out countermeasures to prevent outage. A reported threat of malware targeting online banking applications to steal user credentials and conduct un-authorized financial transactions for example allows the CISO to issue monitoring alerts for the online banking application secure incident event monitoring management team.

Security in SDLC Issue Management Metrics

Once vulnerabilities are identified the next step is to decide which should be fixed and when and how should be fixed. The first question can be answered by the vulnerability assessment process compliance requirements that might require for example of high risk type of vulnerabilities to be remediated in shorter time frames than medium and low risk type of vulnerabilities. The requirement might also vary depending on the type of web application, being for example a totally newly developed web application versus a new release of an existing web application. Since new web applications were not security tested before, they represent higher risks than existing web applications and therefore this might require that high risk vulnerabilities to be mitigated prior to release the application in the production. Once the issues are identified and prioritized for mitigation based upon the risk severity of the vulnerability, the next step is to determine how to fix the vulnerability. This depends on factors such as the type of the vulnerability such as the security controls/measures that are affected by the vulnerability and where the vulnerability is most likely being introduced. This type of metrics allows the CISO to point to the root causes of vulnerabilities and present the case for remediation to the application development teams. When the vulnerability metrics is reported as a trend, it allows the CISO to assess improvements. For example, in the case the same type of security issues are measures over time for the same type of web application, it is possible for the CISO to point to potential root causes. For example, with trend vulnerability metrics and categorization of the type of vulnerabilities, it is possible for the CISO to make the case of investing in certain type of security activities such as process improvements, adoption of testing tools as well as training and awareness. For example, the metrics showed in figure 1 shows positive trends of certain type of vulnerabilities by comparing two quarterly releases of the same web applications. Application security improvements measures as a reduced number of vulnerabilities identified from one quarterly release to another is observed for most vulnerability types except for authentication and user/session management issues.

The CISO might use this metrics to discuss with CIOs and development directors on whether the organization is getting better or worst over time in releasing more secure application software and to direct the application security resources (e.g. process, people and tools) where is most needed for reducing risks. With the metrics shown in Figure 1 for example, assuming the application changes introduced between releases do not differ much in term of type and complexity of the changes introduced as well as the number and the type of software developers in the development team and the tools used, a case can be made on focusing on the type of vulnerabilities that the organization is having trouble fixing such as better design and implementation of authentication and user/session management controls. The CISO might then coordinate with the CIO and the development directors to schedule a targeted training on this type of vulnerabilities, document development guides for authentication and session management and adopt specific security test cases. Ultimately this coordinated effort will empower software developers in designing, implementing and testing more secure authentication and session management controls and show these as improvements in the vulnerability metrics.

Another important aspect of the S-SDLC security metrics is to decide where in the SDLC to invest in security testing and remediation. To know this, it is important to measure in which phase of the SDLC the most of vulnerabilities (higher percentage of issues) originate, when these vulnerabilities are tested and how much cost to the organization to fix them in each phase of the SDLC. A sample metrics that measure this is shown in figure 2 based upon a case study on the costs of testing and managing software bugs (Ref Capers Jones Study). A similar type of security defect management metrics can be used by the CISOs for managing security issues effectively by reducing overall security costs. Assuming the CISO has rolled out a security in the SDLC process and has budget allocated for investment in security in the SDLC activities such as secure coding training and secure code review process and static code analysis tools, this metrics allows the CISO to make that case for investing in testing and fixing security issues in the early phases of the SDLC. This is based upon the following measurements from this case study: 1) most of the vulnerabilities are introduced by software developers during coding, 2) the majority of these vulnerabilities are tested during field tests prior to production and 3) testing and fixing vulnerabilities late in the SDLC is the most inefficient way to do it since is approx. ten times more expensive to fix issues during pre-production tests than during unit tests. CISOs can use vulnerability case studies like these or use their own metrics to make the case for investing in secure software development activities since these will save the organization time and money.

References

Ref [1]: Verizon 2011 Data Breach Investigation Report: http://www.verizonbusiness.com/resources/reports/rp_data-breach-investigations-report-2011_en_xg.pdf

Ref [2]: US Q2 2011 GDP Report Is Bad News for the US Tech Sector, But With Some Silver Linings: http://blogs.forrester.com/andrew_bartels/11-07-29-us_q2_2011_gdp_report_is_bad_news_for_the_us_tech_sector_but_with_some_silver_linings

Ref [3]: Supplement to Authentication in an Internet Banking Environment: http://www.fdic.gov/news/news/press/2011/pr11111a.pdf

Ref [4] PCI-DSS: https://www.pcisecuritystandards.org/security_standards/index.php

Ref [5] OWASP Top Ten: https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project

Ref [6] Gartner teleconference on application security, Joseph Feiman, VP and Gartner Fellow http://www.gartner.com/it/content/760400/760421/ks_sd_oct.pdf

Ref [7] Identity Theft Survey Report, Federal Trade Commission,September, 2003: http://www.ftc.gov/os/2003/09/synovatereport.pdf

Ref [8] Dan E Geer Economics and Strategies of Data Security: http://www.verdasys.com/thoughtleadership/

Ref [9] Data Loss Database: http://datalossdb.org/

Ref [10] WHID, Web Hacking Incident Database: http://projects.webappsec.org/w/page/13246995/Web-Hacking-Incident-Database

Ref [11] Imperva's Web Application Attack Report: http://www.imperva.com/docs/HII_Web_Application_Attack_Report_Ed1.pdf

Ref [12] Albert Gonzalez data breach indictment: http://www.wired.com/images_blogs/threatlevel/2009/08/gonzalez.pdf

Ref [13] First Annual Cost of Cyber Crime Study Benchmark Study of U.S. Companies, Sponsored by ArcSight Independently conducted by Ponemon Institute LLC, July 2010: http://www.arcsight.com/collateral/whitepapers/Ponemon_Cost_of_Cyber_Crime_study_2010.pdf

Ref [14] 2010 Annual Study: U.S. Cost of a Data Breach: http://www.symantec.com/content/en/us/about/media/pdfs/symantec_ponemon_data_breach_costs_report.pdf?om_ext_cid=biz_socmed_twitter_facebook_marketwire_linkedin_2011Mar_worldwide_costofdatabreach

Ref [15] Gordon, L.A. and Loeb, M.P. “The economics of information security investment”, ACM Transactions on Information and Systems Security, Vol.5, No.4, pp.438-457, 2002.

Ref [16] Total Cost of Ownership: http://en.wikipedia.org/wiki/Total_cost_of_ownership

Ref [17] Wes SonnenReich, Return of Security Investment, Practical Quantitative Model: http://www.infosecwriters.com/text_resources/pdf/ROSI-Practical_Model.pdf

Ref [18] Tangible ROI through Secure Software Engineering: http://www.mudynamics.com/assets/files/Tangible%20ROI%20Secure%20SW%20Engineering.pdf

Ref [19] The Privacy Dividend: the business case for investing in proactive privacy protection, Information Commissioner's Office, UK, 2009: http://www.ico.gov.uk/news/current_topics/privacy_dividend.aspx

Ref[20] Share prices and data breaches: http://www.securityninja.co.uk/data-loss/share-prices-and-data-breaches/

Ref[21] A commissioned study conducted by Forrester Consulting on behalf of VeriSign: DDoS: A Threat You Can’t Afford To Ignore: http://www.verisigninc.com/assets/whitepaper-ddos-threat-forrester.pdf

Ref[22] Sony data breach could be most expensive ever: http://www.csmonitor.com/Business/2011/0503/Sony-data-breach-could-be-most-expensive-ever

Ref[23] Health Net discloses loss of data to 1.9 million customers: http://www.computerworld.com/s/article/9214600/Health_Net_discloses_loss_of_data_to_1.9_million_customers

Ref[24] EMC spends $66 million to clean up RSA SecureID mess: http://www.infosecurity-us.com/view/19826/emc-spends-66-million-to-clean-up-rsa-secureid-mess/

Ref[25] Dmitri Alperovitch, Vice President, Threat Research, McAfee, Revealed: Operation Shady RAT: http://www.mcafee.com/us/resources/white-papers/wp-operation-shady-rat.pdf

Ref[26] OWASP Security Spending Benchmarks Project Report: https://www.owasp.org/images/b/b2/OWASP_SSB_Project_Report_March_2009.pdf

Ref[27] The Security Threat/Budget Paradox: http://www.verizonbusiness.com/Thinkforward/blog/?postid=164

Ref [?] S-SDLC processes: MS-SDL, OWASP-CLASP, Cigital-TP

Ref [?] ESAPI

Ref[?] OWASP-CLASP

Ref[?] OWASP- Development Guide

Ref[?] OWASP- Coding Guide

Ref[?] OWASP- Testing Guide

Ref [?] Security and the Software Development Lifecycle: Secure at the Source, Aberdeen Group, 2011 http://www.aberdeen.com/Aberdeen-Library/6983/RA-software-development-lifecycle.aspx

Ref [?] State of Application Security - Immature Practices Fuel Inefficiencies, But Positive ROI Is Attainable, Forrester Consulting, 2011 http://www.microsoft.com/downloads/en/details.aspx?FamilyID=813810f9-2a8e-4cbf-bd8f-1b0aca7af61d&displaylang=en

About OWASP

Short piece about OWASP and including links to Projects, ASVS, SAMM, Commercial Code of Conduct, Citations, ???

Appendix I-A: Value of Data & Cost of an Incident

The discussion of various information sources, which gives a single illustrative value for the main text

Value of Information

The selection of security measures must consider the value of asset being protected. Like personal data, all types of data can have value determined from a number of different perspectives. While it may be most common the look at the value of data by its value as an asset to the organization or the cost of an incident, these are neither always the most appropriate nor greatest valuations to consider. For example, a report looking at the value of personal data (personally identifiable information) suggests four perspectives from which personal information draws its privacy value (Ref 19). These are:

• its value as an asset used within the organization’s operations;
• its value to the individual to whom it relates;
• its value to other parties who might want to use the information, whether for legitimate or improper purposes;
• its societal value as interpreted by regulators and other groups.

The value to the subject of the data, to other parties or to society may be more appropriate for some organizations than others. The report (ref 19) also examines the wider consequences of not protecting (personal) data and the benefits of protection. It describes how incidents involving personal data that lead to financial fraud can have much larger impacts on individuals, but that financial effects are not the only impact. The report provides methods of calculation, and provides examples where the value of an individual's personal data record could be in the £500-£1,100 (approximately $800-$1,800) in 2008.

Data Breaches and Monetary Losses

Regarding the monetary loss per victim, exact figures vary depending on the factors that are considered to calculate them depending by the type of industry and the type of attack causing the data loss incident. According to a July 2010 study conducted by Ponemon Institute (Ref[13]) on 45 organizations of different industry sectors about the costs of cyber attacks, the costs of web-based attacks is 17% of the annualized cyber-attack costs. This cost varies across different industry sectors with the higher costs for defense, energy and financial services ($16.31 million, $15.63 million and $12.37 million respectively) than organization in retails, services and education.

Also according to the 2011 Ponemon Institute annual survey of data loss costs for U.S. companies (Ref[14]), the average cost per compromised record in 2010 was $214 up 5% from 2009. According to this survey, the communication sector bear the highest cost of $380 per customer record with financial services the second highest cost of $353 followed by healthcare with $345, media, at $131, education at $112 and the public sector at $81.

The security company Symantec, which sponsored the report, developed with Ponemon Institute a data breach risk calculator that can be used to calculate the likelihood of data breach in the next 12 months, as well as to calculate the the average cost per breach and average cost per lost record.

The Ponemon institute direct costs estimates, are also used for estimating the direct cost of data breach incidents collected by OSF DataLossDB (Ref[9]). 2009 direct cost figures of $60.00/record are multiplied by the number of records reported by each incident to obtain the monetary loss estimate. It is assumed that direct costs are suffered by the breached organizations while this is not always true such as in the case of credit card number breaches where the direct costs can often be suffered by banks and card issuers. Furthermore, estimate costs does not include indirect costs (e.g. time, effort and other organizational resources spent) as well as opportunity costs (e.g. the cost resulting from lost business opportunities because of reputational damage).

According to (Ref [8]) another possible ways to make a risk management decision on whether to mitigate a potential loss is to determine if the company will be legally liable for that data loss. By using the definition of legal liability from a U.S. liability case law, given as Probability (P) of the loss, (L) the amount of the Loss, then there is liability whenever the cost of adequate precautions or the Burden (B) to the company is:

      B < P x L

By applying this formula to 2003 data from the the Federal Trade Commission (FTC) for example (Ref [7]), the probability of the loss is 4.6% as the amount of the population that suffered identity fraud while the amount of the loss x victim can be calculated by factoring how much money was spent to recover from the loss considering the time spent was 300 million hours at the hourly wages of $ 5.25/hr plus out of pocket expenses of $ 5 billion:

     L = [Time Spent x Recover From Loss x Hourly Wage + Out Of Pocket Expenses]/Number of Victims

With this formula for calculating the amount of loss due to an identity fraud incident, based upon 2003 FTC data, the loss per customer/victim is approximately $ 655 dollars and the burden imposed to the company is $ 30.11 per customer/victim per incident.

The risk management decision is then to decide to whether it is possible to protect a customer for $ 30.11 per customer per annum. If it is, according to (Ref [8]) then liability is found and there is liability risk for the company. This calculation can be useful to determine the potential liability risk in case of data loss incidents, for example by applying the FTC figures to the TJX Inc. incident of 2007 where it was initially announced the exposure of confidential information of 45,700,000 customers, the exposure to the incident for the victims involved could be calculated as:

    Cost exposure to the incident = Number of victims exposed by the incident x loss per victim

With this formula using TJX Inc data or number of victims affected and by applying the loss per victim using FTC data, the cost of the incident that represents the loss potential is $ 30 Billion. By factoring this with the probability of the incident occurring, then it is possible to determine how much money should be spend in security measures. In the case of TJX Inc incident for example, assuming a 1 in 1000 chance of occurrence a $ 30 Million security program for TJX Inc would have been justifiable.

Summary

We can see that there are different ways to determine the value of information and the that some of these are purely based on the costs relating to data breaches. But overall, the references suggest that typically individual's data can be valued in the range $500 to $2,000 per record.

Appendix I-B: Calculation Sheets

Some grids for CISOs to enter their own numbers and calculations

Appendix I-C: Online Calculator

A calculator for estimating the cost incurred by organizations, across industry sectors, after experiencing a data breach is provided by Symantec based upon data surveys of the Ponemon institute: https://databreachcalculator.com/

This article is a stub. You can help OWASP by expanding it or discussing it on its Talk page.