The OWASP Testing Framework - Revision history

Daniel.schulz: Added external link

2016-06-15T20:57:36Z

Added external link

Jane O'Connor: Added Andrews changes

2014-07-27T17:26:56Z

Added Andrews changes

Jane O'Connor at 07:03, 13 May 2014

2014-05-13T07:03:05Z

Jane O'Connor: Final edit

2014-05-13T06:59:54Z

Final edit

Show changes

Andrea Rosignoli at 13:58, 4 February 2014

2014-02-04T13:58:44Z

David Fern: Updated link

2012-11-06T15:40:26Z

Updated link

KirstenS: /* Phase 1A: Review Policies and Standards */

2009-02-02T13:27:51Z

‎Phase 1A: Review Policies and Standards

KirstenS: /* Overview */

2009-02-02T13:27:16Z

‎Overview

KirstenS at 13:44, 26 November 2008

2008-11-26T13:44:23Z

Namn: /* Phase 2B: Design and Architecture Review */

2008-09-10T07:01:01Z

‎Phase 2B: Design and Architecture Review

@@ Line 106: / Line 106: @@
 * Business requirements for availability, confidentiality, and integrity.
 * OWASP Guide or Top 10 Checklists for technical exposures (depending on the depth of the review).
-* Specific issues relating to the language or framework in use, such as the Scarlet paper for PHP or Microsoft Secure Coding checklists for ASP.NET.
+* Specific issues relating to the language or framework in use, such as the Scarlet paper for PHP or [https://msdn.microsoft.com/en-us/library/ff648269.aspx Microsoft Secure Coding checklists for ASP.NET].
 * Any industry specific requirements, such as Sarbanes-Oxley 404, COPPA, ISO/IEC 27002, APRA, HIPAA, Visa Merchant guidelines, or other regulatory regimes.
 In terms of return on resources invested (mostly time), static code reviews produce far higher quality returns than any other security review method and rely least on the skill of the reviewer. However, they are not a silver bullet and need to be considered carefully within a full-spectrum testing regime.
 For more details on OWASP checklists, please refer to [[OWASP Guide Project|OWASP Guide for Secure Web Applications]], or the latest edition of the [[OWASP Top 10]].
 ==Phase 4: During Deployment==

@@ Line 4: / Line 4: @@
 This section describes a typical testing framework that can be developed within an organization. It can be seen as a reference framework that comprises techniques and tasks that are appropriate at various phases of the software development life cycle (SDLC). Companies and project teams can use this model to develop their own testing framework and to scope testing services from vendors. This framework should not be seen as prescriptive, but as a flexible approach that can be extended and molded to fit an organization’s development process and culture.
 This section aims to help organizations build a complete strategic testing process, and is not aimed at consultants or contractors who tend to be engaged in more tactical, specific areas of testing.
 It is critical to understand why building an end-to-end testing framework is crucial to assessing and improving software security. In ''Writing Secure Code'' Howard and LeBlanc note that issuing a security bulletin costs Microsoft at least $100,000, and it costs their customers collectively far more than that to implement the security patches. They also note that the US government’s CyberCrime web site (<u>http://www.justice.gov/criminal/cybercrime/</u>) details recent criminal cases and the loss to organizations. Typical losses far exceed USD $100,000.
 With economics like this, it is little wonder why software vendors move from solely performing black box security testing, which can only be performed on applications that have already been developed, to concentrate on testing in the early cycles of application development such as definition, design, and development.
 Many security practitioners still see security testing in the realm of penetration testing. As discussed before, while penetration testing has a role to play, it is generally inefficient at finding bugs and relies excessively on the skill of the tester. It should only be considered as an implementation technique, or to raise awareness of production issues. To improve the security of applications, the security quality of the software must be improved. That means testing the security at the definition, design, develop, deploy, and maintenance stages, and not relying on the costly strategy of waiting until code is completely built.
 As discussed in the introduction of this document, there are many development methodologies such as the Rational Unified Process, eXtreme and Agile development, and traditional waterfall methodologies. The intent of this guide is to suggest neither a particular development methodology nor provide specific guidance that adheres to any particular methodology. Instead, we are presenting a generic development model, and the reader should follow it according to their company process.
 This testing framework consists of the following activities that should take place:
@@ Line 61: / Line 55: @@
 When looking for requirements gaps, consider looking at security mechanisms such as:
-* User Management (password reset etc.)
+* User Management
 * Authentication
 * Authorization
@@ Line 70: / Line 64: @@
 * Transport Security
 * Tiered System Segregation
-* Privacy
+* Legislative and standards compliance (including Privacy, Government and Industry standards)
@@ Line 113: / Line 107: @@
 * OWASP Guide or Top 10 Checklists for technical exposures (depending on the depth of the review).
 * Specific issues relating to the language or framework in use, such as the Scarlet paper for PHP or Microsoft Secure Coding checklists for ASP.NET.
-* Any industry specific requirements, such as Sarbanes-Oxley 404, COPPA, ISO 17799, APRA, HIPAA, Visa Merchant guidelines, or other regulatory regimes.
+* Any industry specific requirements, such as Sarbanes-Oxley 404, COPPA, ISO/IEC 27002, APRA, HIPAA, Visa Merchant guidelines, or other regulatory regimes.
 In terms of return on resources invested (mostly time), static code reviews produce far higher quality returns than any other security review method and rely least on the skill of the reviewer. However, they are not a silver bullet and need to be considered carefully within a full-spectrum testing regime.

@@ Line 4: / Line 4: @@
 This section describes a typical testing framework that can be developed within an organization. It can be seen as a reference framework that comprises techniques and tasks that are appropriate at various phases of the software development life cycle (SDLC). Companies and project teams can use this model to develop their own testing framework and to scope testing services from vendors. This framework should not be seen as prescriptive, but as a flexible approach that can be extended and molded to fit an organization’s development process and culture.
 This section aims to help organizations build a complete strategic testing process, and is not aimed at consultants or contractors who tend to be engaged in more tactical, specific areas of testing.
 It is critical to understand why building an end-to-end testing framework is crucial to assessing and improving software security. In ''Writing Secure Code'' Howard and LeBlanc note that issuing a security bulletin costs Microsoft at least $100,000, and it costs their customers collectively far more than that to implement the security patches. They also note that the US government’s CyberCrime web site (<u>http://www.justice.gov/criminal/cybercrime/</u>) details recent criminal cases and the loss to organizations. Typical losses far exceed USD $100,000.
 With economics like this, it is little wonder why software vendors move from solely performing black box security testing, which can only be performed on applications that have already been developed, to concentrate on testing in the early cycles of application development such as definition, design, and development.
 Many security practitioners still see security testing in the realm of penetration testing. As discussed before, while penetration testing has a role to play, it is generally inefficient at finding bugs and relies excessively on the skill of the tester. It should only be considered as an implementation technique, or to raise awareness of production issues. To improve the security of applications, the security quality of the software must be improved. That means testing the security at the definition, design, develop, deploy, and maintenance stages, and not relying on the costly strategy of waiting until code is completely built.
 As discussed in the introduction of this document, there are many development methodologies such as the Rational Unified Process, eXtreme and Agile development, and traditional waterfall methodologies. The intent of this guide is to suggest neither a particular development methodology nor provide specific guidance that adheres to any particular methodology. Instead, we are presenting a generic development model, and the reader should follow it according to their company process.
 This testing framework consists of the following activities that should take place:
@@ Line 29: / Line 35: @@
 Before application development starts an adequate SDLC must be defined where security is inherent at each stage.
 ===Phase 1.2: Review Policies and Standards===
@@ Line 37: / Line 44: @@
 If the application is to be developed in Java, it is essential that there is a Java secure coding standard. If the application is to use cryptography, it is essential that there is a cryptography standard. No policies or standards can cover every situation that the development team will face. By documenting the common and predictable issues, there will be fewer decisions that need to be made during the development process.
 ===Phase 1.3: Develop Measurement and Metrics Criteria and Ensure Traceability===
@@ Line 63: / Line 71: @@
 * Tiered System Segregation
 * Privacy
 ===Phase 2.2: Review Design and Architecture===
@@ Line 71: / Line 80: @@
 If weaknesses are discovered, they should be given to the system architect for alternative approaches.
 ===Phase 2.3: Create and Review UML Models===
 Once the design and architecture is complete, build Unified Modeling Language (UML) models that describe how the application works. In some cases, these may already be available. Use these models to confirm with the systems designers an exact understanding of how the application works. If weaknesses are discovered, they should be given to the system architect for alternative approaches.
 ===Phase 2.4: Create and Review Threat Models===
@@ Line 84: / Line 95: @@
 Theoretically, development is the implementation of a design. However, in the real world, many design decisions are made during code development. These are often smaller decisions that were either too detailed to be described in the design, or issues where no policy or standard guidance was offered. If the design and architecture were not adequate, the developer will be faced with many decisions. If there were insufficient policies and standards, the developer will be faced with even more decisions.
 ===Phase 3.1: Code Walk Through===
@@ Line 90: / Line 102: @@
 The purpose is not to perform a code review, but to understand at a high level the flow, the layout, and the structure of the code that makes up the application.
 ===Phase 3.2: Code Reviews===
@@ Line 111: / Line 124: @@
 Having tested the requirements, analyzed the design, and performed code review, it might be assumed that all issues have been caught. Hopefully this is the case, but penetration testing the application after it has been deployed provides a last check to ensure that nothing has been missed.
 ===Phase 4.2: Configuration Management Testing===
@@ Line 122: / Line 136: @@
 There needs to be a process in place which details how the operational side of both the application and infrastructure is managed.
 ===Phase 5.2: Conduct Periodic Health Checks===
 Monthly or quarterly health checks should be performed on both the application and infrastructure to ensure no new security risks have been introduced and that the level of security is still intact.
 ===Phase 5.3: Ensure Change Verification===

@@ Line 49: / Line 49: @@
 ===Phase 2A: Review Security Requirements===
-Security requirements define how an application works from a security perspective. It is essential that the security requirements be tested. Testing in this case means testing the assumptions that are made in the requirements, and testing to see if there are gaps in the requirements definitions.
+Security requirements define how an application works from a security perspective. It is essential that the security requirements get tested. Testing in this case means testing the assumptions that are made in the requirements, and testing to see if there are gaps in the requirements definitions.
 For example, if there is a security requirement that states that users must be registered before they can get access to the whitepapers section of a website, does this mean that the user must be registered with the system, or should the user be authenticated? Ensure that requirements are as unambiguous as possible.

@@ Line 7: / Line 7: @@
 This section aims to help organizations build a complete strategic testing process, and is not aimed at consultants or contractors who tend to be engaged in more tactical, specific areas of testing.
-It is critical to understand why building an end-to-end testing framework is crucial to assessing and improving software security. Howard and LeBlanc note in ''Writing Secure Code'' that issuing a security bulletin costs Microsoft at least $100,000, and it costs their customers collectively far more than that to implement the security patches. They also note that the US government’s CyberCrime web site (<u>http://www.cybercrime.gov/cccases.html</u>) details recent criminal cases and the loss to organizations. Typical losses far exceed USD $100,000.
+It is critical to understand why building an end-to-end testing framework is crucial to assessing and improving software security. Howard and LeBlanc note in ''Writing Secure Code'' that issuing a security bulletin costs Microsoft at least $100,000, and it costs their customers collectively far more than that to implement the security patches. They also note that the US government’s CyberCrime web site (<u>http://www.justice.gov/criminal/cybercrime/</u>) details recent criminal cases and the loss to organizations. Typical losses far exceed USD $100,000.
 With economics like this, it is little wonder why software vendors move from solely performing black box security testing, which can only be performed on applications that have already been developed, to concentrate on the early cycles of application development such as definition, design, and development.

@@ Line 35: / Line 35: @@
 Ensure that there are appropriate policies, standards, and documentation in place. Documentation is extremely important as it gives development teams guidelines and policies that they can follow.
-''People can only do the right thing, if they know what the right thing is.''' '''''
+''People can only do the right thing if they know what the right thing is.''' '''''
 If the application is to be developed in Java, it is essential that there is a Java secure coding standard. If the application is to use cryptography, it is essential that there is a cryptography standard. No policies or standards can cover every situation that the development team will face. By documenting the common and predictable issues, there will be fewer decisions that need to be made during the development process.

@@ Line 17: / Line 17: @@
 This testing framework consists of the following activities that should take place:
-* Before Development Begins
+* Before development begins
-* During Definition and Design
+* During definition and design
-* During Development
+* During development
-* During Deployment
+* During deployment
-* Maintenance and Operations
+* Maintenance and operations
 ==Phase 1: Before Development Begins==

@@ Line 11: / Line 11: @@
 With economics like this, it is little wonder why software vendors move from solely performing black box security testing, which can only be performed on applications that have already been developed, to concentrate on the early cycles of application development such as definition, design, and development.
-Many security practitioners still see security testing in the realm of penetration testing. As discussed before, while penetration testing has a role to play, it is generally inefficient at finding bugs and relies excessively on the skill of the tester. It should only be considered as an implementation technique, or to raise awareness of production issues. To improve the security of applications, the security quality of the software must be improved. That means testing the security at the definition, design, develop, deploy, and maintenance stages, and not relying on the costly strategy of waiting until code is completely built.
+Many security practitioners still see security testing in the realm of penetration testing. As discussed before, while penetration testing has a role to play, it is generally inefficient at finding bugs, and relies excessively on the skill of the tester. It should only be considered as an implementation technique, or to raise awareness of production issues. To improve the security of applications, the security quality of the software must be improved. That means testing the security at the definition, design, develop, deploy, and maintenance stages, and not relying on the costly strategy of waiting until code is completely built.
 As discussed in the introduction of this document, there are many development methodologies such as the Rational Unified Process, eXtreme and Agile development, and traditional waterfall methodologies. The intent of this guide is to suggest neither a particular development methodology nor provide specific guidance that adheres to any particular methodology. Instead, we are presenting a generic development model, and the reader should follow it according to their company process.
@@ Line 70: / Line 70: @@
 Applications should have a documented design and architecture. By documented, we mean models, textual documents, and other similar artifacts. It is essential to test these artifacts to ensure that the design and architecture enforce the appropriate level of security as defined in the requirements.
-Identifying security flaws in the design phase is not only one of the most cost efficient places to identify flaws, but can be one of the most effective places to make changes. For example, if it is identified that the design calls for authorization decisions to be made in multiple places, it may be appropriate to consider a central authorization component. If the application is performing data validation at multiple places, it may be appropriate to develop a central validation framework (fixing input validation in one place, rather than in hundreds of places, is far cheaper).
+Identifying security flaws in the design phase is not only one of the most cost-efficient places to identify flaws, but can be one of the most effective places to make changes. For example, if it is identified that the design calls for authorization decisions to be made in multiple places, it may be appropriate to consider a central authorization component. If the application is performing data validation at multiple places, it may be appropriate to develop a central validation framework (fixing input validation in one place, rather than in hundreds of places, is far cheaper).
 If weaknesses are discovered, they should be given to the system architect for alternative approaches.

@@ Line 66: / Line 66: @@
 * Privacy
-===Phase 2B: Design and Architecture Review===
+===Phase 2B: Review Design and Architecture===
 Applications should have a documented design and architecture. By documented, we mean models, textual documents, and other similar artifacts. It is essential to test these artifacts to ensure that the design and architecture enforce the appropriate level of security as defined in the requirements.