This site is the archived OWASP Foundation Wiki and is no longer accepting Account Requests.
To view the new OWASP Foundation website, please visit https://owasp.org
Difference between revisions of "Community Engagement Results QA Testing 2014"
(→Results of Activity) |
(→October- November 2014) |
||
(34 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
=Origin of Activity= | =Origin of Activity= | ||
In 2014, the board decided to downgrade all the flagship projects at the time to LAB status: | In 2014, the board decided to downgrade all the flagship projects at the time to LAB status: | ||
+ | http://lists.owasp.org/pipermail/owasp-board/2014-May/013788.html | ||
+ | |||
+ | Announced on the Project Task Force mailing list: | ||
+ | |||
https://groups.google.com/forum/#!topic/owasp-projects-task-force/X2b9J1eSC5E | https://groups.google.com/forum/#!topic/owasp-projects-task-force/X2b9J1eSC5E | ||
− | (need an owasp account to | + | Some issues regarding this decision in the mailing list: |
+ | |||
+ | *http://lists.owasp.org/pipermail/owasp-testing/20140605/002322.html | ||
+ | *http://damonmiller513.blogspot.com/2014/04/my-thoughts-on-inevitable-esapi-demotion.html | ||
+ | |||
+ | Official letter by the board (need an owasp account to google doc): | ||
+ | |||
+ | http://lists.owasp.org/pipermail/owasp-board/2014-May/013788.html | ||
https://docs.google.com/a/owasp.org/document/d/1KGwq6dT5LWfRPUfmSLD-ZPxGJcWFcoOrvgPOjKFFVY8/edit?usp=sharing | https://docs.google.com/a/owasp.org/document/d/1KGwq6dT5LWfRPUfmSLD-ZPxGJcWFcoOrvgPOjKFFVY8/edit?usp=sharing | ||
− | =Action== | + | ==Action taken== |
− | Budget was provided to major test all demoted projects | + | Budget was provided to major test all demoted projects to the task force team after the following proposal and discussions on the project leaders mailing list |
+ | https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach | ||
+ | |||
+ | Proposal redifinition committee: | ||
+ | [[File:Committee_2-project_reviews.pdf]] | ||
+ | |||
+ | Actions were taken to look for testers and developments regarding this: | ||
+ | |||
+ | *http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-May/000212.html | ||
+ | *http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-June/000217.html | ||
+ | *http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-June/000218.html | ||
+ | |||
+ | After a search , we decided (Board+ task force) to hire Marios Kourtesis which had an excellent background in security and using open source projects. Also he could provide the service for a lower price that the major offers done by freelance companies and services | ||
==Proposal for testing== | ==Proposal for testing== | ||
Line 16: | Line 39: | ||
A budget of USD7000- was set aside for this purpose however only half was used thanks to donations: | A budget of USD7000- was set aside for this purpose however only half was used thanks to donations: | ||
Original proposal: | Original proposal: | ||
− | + | https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach | |
Community fund allocation as appears today. I would like to see this corrected | Community fund allocation as appears today. I would like to see this corrected | ||
https://www.owasp.org/index.php/Community_Engagement_-_Payments#2015_Community_Engagement_Allocations.2FPayments | https://www.owasp.org/index.php/Community_Engagement_-_Payments#2015_Community_Engagement_Allocations.2FPayments | ||
− | Jira account has been donated by Atlassian (thanks to Norman Yue) | + | ===Proposed Budget=== |
− | Ranorex Tool (1) license ==> USD 2,706.4(was donated by Ranorex) | + | *Jira account has been donated by Atlassian (thanks to Norman Yue) |
− | 2 Virtual Servers(1 Linux/1 Windows) (Leaseweb) USD 130/server for 6 months period per server/ ==> USD 262,- | + | *Ranorex Tool (1) license ==> USD 2,706.4(was donated by Ranorex) |
− | 1 tester @USD25/hour ==> Maximum hours 160==> Total USD 4,000- | + | *2 Virtual Servers(1 Linux/1 Windows) (Leaseweb) USD 130/server for 6 months period per server/ ==> USD 262,- |
− | Total budget: USD 6968.4,- | + | *1 tester @USD25/hour ==> Maximum hours 160==> Total USD 4,000- |
+ | Total budget proposed: USD 6968.4,- | ||
− | ==Actual budget used== | + | ===Actual budget used=== |
JIRA and Ranorex sponsored with license(no cost) | JIRA and Ranorex sponsored with license(no cost) | ||
Line 34: | Line 58: | ||
We had a VM machine with Jenkins/TeamCity server for a period of one year USD750 (setup by Jason Johnson) | We had a VM machine with Jenkins/TeamCity server for a period of one year USD750 (setup by Jason Johnson) | ||
− | + | In the end we used half of this budget | |
− | Payment to tester (98 hours) = USD2450,- | + | *Payment to tester (98 hours) = USD2450,- |
− | Payment VM machine Jenkings = USD750 | + | *Payment VM machine Jenkings = USD750 |
− | + | *Payment VM machine for manual testing(Ranorex tool installed) =USD262 | |
+ | *Use the SWAMP for testing builds (this service is free) | ||
Total actually used:~USD3500 | Total actually used:~USD3500 | ||
==Results of Testing and further developments == | ==Results of Testing and further developments == | ||
− | ===October- November 2014== | + | ===October- November 2014=== |
We used some of the reviews done in 2013 by the Advisor team: | We used some of the reviews done in 2013 by the Advisor team: | ||
− | http://owasp.blogspot.com/2013/09/meet-our-new-technical-project-advisors.html | + | *http://owasp.blogspot.com/2013/09/meet-our-new-technical-project-advisors.html |
− | https://drive.google.com/folderview?id=0B1lOCxlYdf1AeUwzWlFfeWg0Mmc&usp=gmail | + | Reviews Advisor team & Criteria: https://drive.google.com/folderview?id=0B1lOCxlYdf1AeUwzWlFfeWg0Mmc&usp=gmail |
− | + | Johanna Curiel did together a major review with Marios Kourtesis and other members , using this information of the reviews in 2013 for the major reviews done in 2014. Jason Johnson help us setting up also a VM with automated Jenkings so project leaders could build their projects automatically and check for errors, Johanna also did some setup projects in this machine which we discontinue using afterwards due to maintenance issues. There were leaders using this tool and testers(Marios & Johanna) on another VM machine for testing manually the projects also using Ranorex which we setup for 3 months that we worked on testing. | |
− | Norman Yuen helped us get a JIRA for better review process and communication | + | Norman Yuen helped us get a JIRA for better review process and communication: |
+ | http://owasporg.atlassian.net | ||
Results of the testing done by Johanna Curiel and Marios Kourtesis | Results of the testing done by Johanna Curiel and Marios Kourtesis | ||
https://www.owasp.org/index.php/LAB_Projects_Code_Analysis_Report | https://www.owasp.org/index.php/LAB_Projects_Code_Analysis_Report | ||
+ | |||
+ | [[File:Image.png | 400px]] | ||
+ | [[File:Screenshot_2015-09-06_14.55.16.png | 400px]] | ||
+ | [[File:Image-2.png| 400px]] | ||
+ | [[File:Image-4.png| 400px]] | ||
==Openduck automation== | ==Openduck automation== | ||
− | We did a major automation using Openduck. We set in total 80 projects that were not registered: | + | We did a major automation using Openduck. We set in total +80 projects that were not registered: |
From there on was easier maintenance which Kait-Disney helped us review the basic criteria, managed the communication with project leaders, maintenance and setup with her the projects in Openduck for automated tracking: | From there on was easier maintenance which Kait-Disney helped us review the basic criteria, managed the communication with project leaders, maintenance and setup with her the projects in Openduck for automated tracking: | ||
https://www.openhub.net/orgs/OWASP | https://www.openhub.net/orgs/OWASP | ||
Line 61: | Line 92: | ||
==2015 - Present== | ==2015 - Present== | ||
Kate helped us as support staff when Samantha and Kait-Disney stopped as staff. She helped on the major part of setting and helping new project leaders. | Kate helped us as support staff when Samantha and Kait-Disney stopped as staff. She helped on the major part of setting and helping new project leaders. | ||
− | + | Johanna Curiel and other members like Timo Goosen helped review some projects that required. Other members contributed with their input on the list. | |
https://www.owasp.org/index.php/Category:OWASP_Project#tab=Project_Task_Force | https://www.owasp.org/index.php/Category:OWASP_Project#tab=Project_Task_Force | ||
Latest revision as of 20:33, 6 September 2015
Origin of Activity
In 2014, the board decided to downgrade all the flagship projects at the time to LAB status: http://lists.owasp.org/pipermail/owasp-board/2014-May/013788.html
Announced on the Project Task Force mailing list:
https://groups.google.com/forum/#!topic/owasp-projects-task-force/X2b9J1eSC5E
Some issues regarding this decision in the mailing list:
- http://lists.owasp.org/pipermail/owasp-testing/20140605/002322.html
- http://damonmiller513.blogspot.com/2014/04/my-thoughts-on-inevitable-esapi-demotion.html
Official letter by the board (need an owasp account to google doc):
http://lists.owasp.org/pipermail/owasp-board/2014-May/013788.html https://docs.google.com/a/owasp.org/document/d/1KGwq6dT5LWfRPUfmSLD-ZPxGJcWFcoOrvgPOjKFFVY8/edit?usp=sharing
Action taken
Budget was provided to major test all demoted projects to the task force team after the following proposal and discussions on the project leaders mailing list https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach
Proposal redifinition committee: File:Committee 2-project reviews.pdf
Actions were taken to look for testers and developments regarding this:
- http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-May/000212.html
- http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-June/000217.html
- http://lists.owasp.org/pipermail/owasp_project_leader_list/2014-June/000218.html
After a search , we decided (Board+ task force) to hire Marios Kourtesis which had an excellent background in security and using open source projects. Also he could provide the service for a lower price that the major offers done by freelance companies and services
Proposal for testing
A proposal was submitted for testing the projects using a clear criteria and QA approach https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach
Budget used
A budget of USD7000- was set aside for this purpose however only half was used thanks to donations: Original proposal: https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach
Community fund allocation as appears today. I would like to see this corrected https://www.owasp.org/index.php/Community_Engagement_-_Payments#2015_Community_Engagement_Allocations.2FPayments
Proposed Budget
- Jira account has been donated by Atlassian (thanks to Norman Yue)
- Ranorex Tool (1) license ==> USD 2,706.4(was donated by Ranorex)
- 2 Virtual Servers(1 Linux/1 Windows) (Leaseweb) USD 130/server for 6 months period per server/ ==> USD 262,-
- 1 tester @USD25/hour ==> Maximum hours 160==> Total USD 4,000-
Total budget proposed: USD 6968.4,-
Actual budget used
JIRA and Ranorex sponsored with license(no cost) We hired one tester for only half the time we budget(attached break down of hours tested, we only use 98 hours) Attached hours testing the builds of projects by tester: https://docs.google.com/a/owasp.org/spreadsheets/d/1_WRxMKrjbVLfctcQcg6oxAc8fymcqOfb7m3tVrB4Hsw/edit?usp=sharing
We had a VM machine with Jenkins/TeamCity server for a period of one year USD750 (setup by Jason Johnson) In the end we used half of this budget
- Payment to tester (98 hours) = USD2450,-
- Payment VM machine Jenkings = USD750
- Payment VM machine for manual testing(Ranorex tool installed) =USD262
- Use the SWAMP for testing builds (this service is free)
Total actually used:~USD3500
Results of Testing and further developments
October- November 2014
We used some of the reviews done in 2013 by the Advisor team:
Reviews Advisor team & Criteria: https://drive.google.com/folderview?id=0B1lOCxlYdf1AeUwzWlFfeWg0Mmc&usp=gmail
Johanna Curiel did together a major review with Marios Kourtesis and other members , using this information of the reviews in 2013 for the major reviews done in 2014. Jason Johnson help us setting up also a VM with automated Jenkings so project leaders could build their projects automatically and check for errors, Johanna also did some setup projects in this machine which we discontinue using afterwards due to maintenance issues. There were leaders using this tool and testers(Marios & Johanna) on another VM machine for testing manually the projects also using Ranorex which we setup for 3 months that we worked on testing. Norman Yuen helped us get a JIRA for better review process and communication: http://owasporg.atlassian.net
Results of the testing done by Johanna Curiel and Marios Kourtesis https://www.owasp.org/index.php/LAB_Projects_Code_Analysis_Report
Openduck automation
We did a major automation using Openduck. We set in total +80 projects that were not registered: From there on was easier maintenance which Kait-Disney helped us review the basic criteria, managed the communication with project leaders, maintenance and setup with her the projects in Openduck for automated tracking: https://www.openhub.net/orgs/OWASP
2015 - Present
Kate helped us as support staff when Samantha and Kait-Disney stopped as staff. She helped on the major part of setting and helping new project leaders. Johanna Curiel and other members like Timo Goosen helped review some projects that required. Other members contributed with their input on the list. https://www.owasp.org/index.php/Category:OWASP_Project#tab=Project_Task_Force
Results 2015
- Activity level monitoring: (Last updated source code repositories including new projects) During the months August 2014 and present, projects are being added to the Openhub repository for monitoring: https://www.openhub.net/orgs/OWASP
- Summary Report: https://magic.piktochart.com/output/6323285-project-reviews
- Some latest summary review results through the Dashboard: https://docs.google.com/a/owasp.org/spreadsheets/d/15NzgmnxKNtexRDs70rBUi1NHhjQiviBdYUa_kDvd3i4/edit?usp=sharing