Project Reviews Guideline

• Main
• Basic Instructions
• Code Criteria
• Tool Criteria
• Health Criteria
• Documentation Criteria
• Determining Flagship status
• Determining LAB status
• Want a review of your project?

Purpose Project Reviews is a process within OWASP to help evaluate the health and quality of OWASP projects. The evaluation is based on a defined criteria which attempts to find out the progress and at which stage of development are the projects. This is the original plan https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach Background Projects are divided in 3 main categories: Code Tools Documentation These are the 3 main development classifications Incubators LAB Flagship Quality of a Code/Tool projects This kind of evaluation requires more work. It is necessary to download, install and smoke test the project. The criteria to evaluate the minimum quality of a project is very simple: For Code and Tools For projects holding Flagship status, we closely monitor their health every 6 months on the following, among other key indicators: Can the project be built correctly? Does the project has any activity(commits) in the last 6 months? Does the project had any releases in the last 6 months? Has the project leaders updated his wiki or website to reflect latest releases? For Documentation For this part, we are working on the development of an adequate assessment criteria The following is a draft of the new process proposal: [Proposal for Reviewing OWASP Document projects]

Presentation https://soundcloud.com/owasp-podcast/owasp-project-reviews-with-johanna-curiel Project Review Team Project Coordinator: Claudia.Aviles-Casanovas Thank you to our season reviewers such as: Thank you to Norman Yue who helped us acquired the JIRA for project reviews Openhub About the Black Duck Open Hub The Black Duck Open Hub (formerly Ohloh.net) is an online community and public directory of free and open source software (FOSS), offering analytics and search services for discovering, evaluating, tracking, and comparing open source code and projects. Open Hub Code Search is free code search engine indexing over 21,000,000,000 lines of open source code from projects on the Black Duck Open Hub. Use Openhub to have an overview of OWASP code and tools activity levels [1] Email List Project Email List https://groups.google.com/a/owasp.org/forum/?hl=en#!forum/projects-task-force

We need regular or season reviewers to help us evaluate projects.

This is how you can help us evaluate the health of a project

• Check the criteria that applies for each project (see tabs for criteria)

Follow the criteria with these instructions:

• Visit the projects inventory page: https://www.owasp.org/index.php/Category:OWASP_Project#tab=Project_Inventory
• Select the project you want to review or the one assigned to you
• Click on the project wiki page you want to review
• Check the history /last updated date and by who
• Check the amount of views on the bottom of the page
• Read the content and click all hyperlinks available, are there any broken?
• Check and read the mailing list and the subjects discussed
• Visit the repository's project (like Github) and check the wiki, issues, branches
• Check if the repository is the same as the one in openhub
• Create a new Google sheet
• Do some google search on the project and the project leader. Make annotations and provide hyperlinks of information found
• Fill in the sheet. Try to add to your report also some print screens, hyperlinks, in other words how did you came to the conclusions and findings in the comments sections. It is very important that you can sustain the findings provided in your report sheet
• share the sheet through the Project task force mailing list

• Does the project have a publicly accessible bug tracking system established, and source code repository?
• Does the project include online documention built into the tool?
• Does the project include build scripts that facilitate building the application from source?
• Does this project have an easy to use installer (Goal: Fully automated installer) (or stand alone executable version)?
• Is the tool/deliverable user friendly and easy to use?
• Smoke test some basic functionalities: Does the project do what they claim? (This might required more a simple smoke test)

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

• Does the project have a publicly accessible bug tracking system established, and source code repository?
• Does the project include online documentation built into the library?
• Does the project include build scripts that facilitate building/adding to the application from source?
• Does this project have an easy to use installer (Goal: Fully automated installer) (or stand alone executable version)?
• Is the library/deliverable user friendly and easy to use?
• Smoke test some basic functionalities: Does the project do what they claim? (This might required more a simple smoke test)

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

For general health criteria we use the following:

Does it meet quality expectations?

• Does the project have a relevant project summary that can be found on the OWASP Project wiki page?

Check the Project wiki page description and introduction sections

• Does the project have a relevant project Roadmap that can be found on the OWASP Project wiki page?

Check a tab call Roadmap, see if there any sore of planning or projection on releases or deliverables

• Does the project have a good track record of resolving issues and answering questions from project consumers?

The best place to check this is the repository issues and wiki page of the projects. Does the project have one? Is it easy to find? How many issues are open/closed?

Does it follow OWASP Project best practices?

• Does the project use an appropriate Community Friendly License?

The project wiki page should contain a description with the type of license provided

• Are project deliverables, information, and releases readily available and accessible to the public?

Does the project have a release version?

• Do the project leaders and contributors perform their duties in accordance to applicable laws?

This is very difficult to asses but try using Google search and finding information about the leaders of the project

Does it support the OWASP mission and objectives?

• Do the project leaders and contributors treat everyone with respect and dignity?

This is very difficult to asses but try using Google search and finding information about the leaders of the project. The project mailing lists history is a good source for this information

• Is the project vendor neutral?

Check for things like Logos in their wiki page or repository, mentioning of commercial activities, logos of vendors in their presentation

• Is the project free and open and not-for-profit?

again , difficult to asses , but try researching through google and find if in any form the leaders are commercial exploiting directly the project by charging users in any form

Does the project have one accepted OWASP reviewed deliverable on record within the new project’s infrastructure? Check previous reviews of projects here: https://docs.google.com/a/owasp.org/spreadsheets/d/15NzgmnxKNtexRDs70rBUi1NHhjQiviBdYUa_kDvd3i4/edit?usp=sharing

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

This is a basic check up we do on Documents:

• Does the project have a publicly accessible bug tracking system established, and source code repository?
• Is the document in a format which can be converted to an OWASP book?
• Does the project release/deliverable have a table of contents that links all the wiki content together?
• Is the project release/deliverable available for download on the OWASP Project wiki page?
• Has all release/deliverable content been reviewed by a technical editor to ensure that English grammar is correct, understandable, and the content flows well?

For this part, we are working on the development of an adequate assessment criteria The following is a draft of the new process proposal: [Proposal for Reviewing OWASP Document projects]

The OWASP Flagship designation is given to projects that have demonstrated superior maturity, established quality, and strategic value to OWASP and application security as a whole. Eligible projects are selected from the OWASP Labs project pool. This selection process generally ensures that there is only one project of each type covering any particular security space. OWASP Flagship projects represent projects that are not only mature, but are also projects that OWASP as an organization provides direct support to maintaining. The core mission of OWASP is to make application security visible and so as an organization, OWASP has a vested interest in the success of its Flagship projects. Since Flagship projects have such high visibility, these projects are expected to uphold the most stringent requirements of all OWASP Projects.

The project must fill all of the following criteria to become a OWASP Flagship project. It is essential for a OWASP Flagship project to have:

• Considerable number of users and contributors
• Considerable number of commits and improvements in a time span of at least two years
• A unique approach or proposition in application security
• Exposure through security conferences
• Use and acceptance by the community
• Being used as reference in books and other resources

OWASP Labs projects represent projects that have delivered significant value. Leaders of OWASP Labs projects are expected to stand behind the quality of their projects as these projects have matured to the point where they are accepted by a significant portion of the OWASP community. While these projects are typically not production ready, the OWASP community expects that an OWASP Labs project leader is producing releases that are ready for mainstream usage. OWASP Labs projects are meant to be a collection of established projects that have gained community support and acclaim by undergoing the project review process.

An OWASP project must fill all of the following criteria to become an OWASP Labs project. It is essential for an OWASP Labs project to have:

• A version number with a clear release schedule
• GitHub source control and a public issue tracking system
• Stable build and release
• Instructions on how to use and build the project properly

Contact Claudia.Aviles-Casanovas[at]owasp.org for requesting more information regarding the process

Fill the following form http://www.tfaforms.com/264413