Benchmark

• Main
• Roadmap

OWASP WebGoat Benchmark The OWASP WebGoat Benchmark is a Java test suite designed to verify the speed and accuracy of vulnerability detection tools. The initial version is intended to support Static Analysis Security Testing Tools (SAST) and Interactive Analysis Security Testing Tools (IAST). A future release (the year hopefully) will suport Dynamic Analysis Security Testing Tools (DAST), like OWASP ZAP. The goal is that the application is fully runable and all the vulnerabilities are actually exploitable so its a fair test for any kind of vulnerability detection tool. Future versions could support other languages, but one step at a time! Project Philosophy Security tools (SAST, DAST, and IAST) are amazing when they find a complex vulnerability in your code. But they can drive everyone crazy with complexity, false alarms, and missed vulnerabilities. We are on a quest to see just how good these tools are at discovering and properly diagnosing security problems in applications. So the WebGoat Benchmark test suite is designed to test the abilities of these tools and hopefully help them improve. Approach For this Java test suite: Every test case is a servlet or JSP Every test case is either a true vulnerability or a false positive for a single issue Metadata for each test case, including expected result is contained in a matching XML file The overall score for a tool is the average of the scores across all the test categories There are two types of tests in the current benchmark: 1) Tests written by hand. 2) Tests that are generated from code snippets that include 3 parts a) A source of taint (e.g., a web parameter) b) Propagation or data flow c) A dangerous sink There are currently over 150 test cases written by hand, and over 20,000 generated tests. Test Coverage For the test suite, we plan to determine, does the tool: Detect the vulnerability? Not detect false alarms? Find HTTP request and response problems? Handle scenarios like: - Simple and complex data flow? - Simple and complex control flow? - Popular frameworks? - Inversion of control? - Reflection? Class loading? Annotations? Future enhancements could cover: Does the tool find flaws in libraries? Does the tool find flaws spanning custom code and libraries? Does tool handle web services? REST, XML, GWT, etc… Does tool work with different app servers? Java platforms? Code Repo The code for this project is hosted at the OWASP Git repository (exact location TBD). Along with the code comes a Maven pom.xml file so you can build the entire project with ease. Licensing The OWASP WebGoat Benchmark is free to use under the TBD license. Mailing List OWASP WebGoat Benchmark Mailing List Project Leaders Dave Wichers @ Related Projects WebGoat NSA's Juliet for Java WAVESEP

Quick Download TBD News and Events [Apr 2015] Initial Release Classifications New BSD License

2015 Roadmap

• [June 2015] TBD
• Analysis tool integration: So you can automaticaly run tools against the benchmark. We want to build test harnesses for tools like:

 * OWASP's ZAP
 * Findbugs and plugins for it like FindSecurityBugs
 * Commercial SAST, DAST, and IAST tools

• FUTURE: Expand to include attack test cases to verify whether defenses (WAF, IDS/IPS, RASP) can identify and protect against them