Difference between revisions of "Assessing Project Releases"

Latest revision as of 14:56, 2 July 2009

This page is maintained by the Global Projects Committee to help assist Project Leaders with information about successfully running an OWASP Project. It will be updated from time to time, and changes will be discussed and announced on the OWASP-Leaders list.

Quality Levels

For project releases, OWASP has created a criteria with three designations of quality: Alpha, Beta and Stable releases. As project releases move up the quality ladder from Alpha to Beta and finally to a Stable release, the amount of rigour required increases. In general, the project lead will determine the goal quality level of their project and work towards fulfilling the criteria for that level. Once a project lead has completed the prerequisites and criteria for the goal level, they request that their project be reviewed. The quality level will determine who reviews the release and how those reviews occur.

• Alpha release: The review consists of the Global Project Committee (GPC) verifying that the project pre-assessment checklist is complete. Alpha release projects are the easiest to achieve since anyone with a start on a solution to an application security problem can self assess their project against the pre-assessment checklist.
• Beta release: The project lead completes the pre-assessment checklist. Then, the review will first be conducted by the project's reviewer (more on this below). After the reviewer completes the review of the release, the GPC will validate the project's review.
• Stable release: The project lead completes the pre-assessment checklist. Then, the two project reviewers will complete their review of the release (more on this below). After the reviews are complete, the Global Projects Committee and OWASP Board will validate the project's review.

Pre-Assessment Checklists versus Reviewer Action Items:

• Pre-Assessment checklists should be completed by the project lead prior to asking for an assessment.
• Pre-Assessment checklists were designed to be completed in minutes to verify that all items are complete.
• Reviewer Action Items should be completed by project reviewer(s) after the project lead indicates the project is ready to be assessed and the pre-assessment checklist is complete.
• Reviewer Action Items were designed to require some significant time commitment from the reviewer since the questions are subjective and require a good deal of understanding and review of the project's release.

Prerequisites for Project Release Assessment

Depending on the quality level criteria, the project lead may have prerequisites to complete before the project release(s) can be assessed by the criteria below

• Alpha release: No prerequisites.
• Beta release: 1 reviewer is required.
• Stable release: 2 reviewers are required. Second review has special requirements.

Notes on reviewers

• Ideally, per project release, the project leader will propose the reviewer(s).
• Ideally, reviewers should be an existing OWASP project leader or chapter leader.
• If the project lead is unable to find the required reviewer(s), the Global Projects Committee can assist in identifying reviewers for the project.
• It is recommended that an OWASP board member or Global Projects Committee member be the second reviewer on Stable releases. The board has the initial option to review the project, followed by the Global Projects Committee.
• The Global Projects Committee confirms the assignment of reviewers to a project.
• For special cases (e.g. large documents), multiple reviewers may be utilized to break the review work into smaller units. The over-riding principal is that one set of eye balls will review for Beta and two sets of eye balls will review for Quality. For example, a large project 4 reviewers could be used to do the Stable quality review were each reviewer would be responsible for reviewing approximately 1/2 of the content where 4 x 1/2 = 2.

Star Rating System

Not Reviewed	Alpha Release	Beta Release	Stable Release

Release Assessment Criteria

Specific Release Assessment Criteria for the OWASP Project Types:

• Tool Assessment Criteria
• Documents Assessment Criteria
• Research and Activities Criteria