Difference between revisions of "Section 4: Mitigating the WebGoat lessons"

Revision as of 08:58, 13 November 2008

Project metrics at 50% project completion

See Section 2 for the WebGoat lesson Table of Contents and an overview of the results from doing the WebGoat lessons. Appendix A contains a zip file which is made up of the lesson plans and solutions - in HTML format - which were taken from WebGoat and can be viewed stand-alone.

Out of 51 possible lessons, the following are teaching lessons, not vulnerabilities, and therefore have no context for ModSecurity rules:

• 1.1 Http Basics
• 4.1 Password Strength
• 15.3 Bypass Client Side JavaScript Validation
• 17.1 Create a SOAP Request

Therefore there is a total number of 47 lessons to do; half is 24 so that was the goal of the first 50% of project completion. The lowest hanging fruit was taken first because considerable effort was put into: (1) setup and configuration of the environment; (2) getting familiar with WebGoat and taking all of the lessons; (3) learning ModSecurity (and Remo); (4) re-learning regular expressions; (5) learning Lua script; and (6) developing an efficient work methodology.

The total number of sublessons mitigated by ModSecurity rules: 25 - thereby achieving the goal of at least 50% of sublessons mitigated.

They are:

• Sublesson 1.2
• Sublesson 2.4
• Sublessons 4.2, 4.4, 4.5
• Sublesson 6.1
• Sublessons 8.1, 8.2, 8.4, 8.5, 8.7
• Sublesson 10.1
• Sublessons 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8
• Sublesson 13.1
• Sublessons 15.1, 15.2
• Sublessons 17.3, 17.4

Project metrics at 100% project completion

Based on Reviewer feedback, three (3) of the 4 sublessons that were originally assessed as not being able to be a ModSecurity solution, actually can be. They are:

• 1.1 Http Basics
• 15.3 Bypass Client Side JavaScript Validation
• 17.1 Create a SOAP Request

Fifteen (15) sublessons were solved in the 2nd phase which makes a total of 40 sublessons solved.

Five (5) sublessons do not count because ModSecurity solutions cannot be written for them (further explanation below):

• 3. AJAX Security -> 3.3 Same Origin Policy Protection
• 5. Buffer Overflows (not implemented)
• 4. Authentication Flaws -> 4.1 Password Strength
• 8. Cross-Site Scripting (XSS) -> 8.6 HTTPOnly Test
• 14. Insecure Storage -> 14.1 Encoding Basics

The remaining 7 sublessons are either unable to be solved or can be solved but were complex solutions and weren’t completed because of lack of time. The next section gives further explanation for each sublesson that was not solved.

Therefore, 40 out of 47 WebGoat sublessons were solved, or 85%. The original goal was 90%, or 42 lessons.

See ‘Section 5, The Mitigating Solutions’ for details on each of the ModSecurity solutions new in Phase 2. In chronological order, these 15 sublessons are:

• Sublesson 12. Insecure Communication -> 12.1 Insecure Login
• Sublesson 7. Concurrency -> 7.1 Thread Safety Problem
• Sublesson 7. Concurrency -> 7.2 Shopping Cart Concurrency Flaw
• Sublesson 2. Access Control Flaws -> 2.2 Bypass a Path Based Access Control Scheme
• Sublesson 8. Cross-Site Scripting (XSS) -> 8.3 Stored XSS Attacks
• Sublesson 9. Denial of Service -> 9.1 Denial of Service from Multiple Logins
• Sublesson 3. AJAX Security -> 3.4 DOM Injection
• Sublesson 3. AJAX Security -> 3.5 XML Injection
• Sublesson 3. AJAX Security -> 3.7 Silent Transactions Attacks
• Sublesson 3. AJAX Security -> 3.1 LAB: DOM-Based cross-site scripting
• Sublesson 3. AJAX Security -> 3.6 JSON Injection
• Sublesson 3. AJAX Security -> 3.8 Dangerous Use of Eval
• Sublesson 1. Introduction -> 1.1 Http Basics
• Sublesson 15. Parameter Tampering -> 15.3 Bypass Client Side JavaScript Validation
• Sublesson 17. Web Services -> 17.1 Create a SOAP Request

Note that as the project progressed, solving ModSecurity solutions over and over resulted in standard formats and cleaner output for the solution write-ups and their artifacts (ModSecurity *.conf files, Lua scripts, HTML error files, etc.). For example, to get an example of a Lua script being used, find one later in the project because it will be more refined and clearer to understand.

Sublessons that do not count or were not solved (and why)

Sublessons that do not count:

• 3. AJAX Security -> 3.3 Same Origin Policy Protection

This WebGoat lesson demonstrates a browser's same origin policy protection and as such has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

• 5. Buffer Overflows (not implemented by WebGoat)

• 4. Authentication Flaws -> 4.1 Password Strength

This WebGoat lesson calls a 3rd party website and shows password strength (time to crack a user-supplied password that matches the criteria); as such it has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

• 8. Cross-Site Scripting (XSS) -> 8.6 HTTPOnly Test

See the Mitigating Solutions section for the link to this page.

• 14. Insecure Storage -> 14.1 Encoding Basics

This WebGoat lesson demonstrates encoding and decoding (e.g. Base64, md5, entity, unicode); as such it has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

Note: ModSecurity has many built-in encoding/decoding transformation functions - it supports each encoding/decoding function in this lesson demonstration, and more.

Sublessons that were not solved (but almost all have suggestions):

See the Mitigating Solutions section for the link to these sublessons:

• 2. Access Control Flaws -> 2.3 LAB: Role Based Access Control

• 3. AJAX Security -> 3.2 LAB: Client Side Filtering (no suggestion for solution)

• 3. AJAX Security -> 3.9 Insecure Client Storage

• 16. Session Management Flaws -> 16.1 Hijack a Session

• 16. Session Management Flaws -> 16.2 Spoof an Authentication Cookie

• 16. Session Management Flaws -> 16.3 Session Fixation

• 17. Web Services -> 17.2 WSDL Scanning

Overall strategy

The intention of mitigating the vulnerabilities is to demonstrate the largest variety of mitigating solutions and features of ModSecurity as possible:

• Some lessons are solved using the easiest way possible for convenience (and to count towards achieving the goal of 50% complete!)
• Some lessons are solved by using rules from the core rulesets provided by Breach Security
• Some mitigating solutions are meant to be global, meaning being in effect at all times, like the XSS and Command Injection core rules from Breach Security
• One lesson demonstrates the use of a session variable
• Some lessons require persistence beyond what is offered by ModSecurity; Lua scripts are used to achieve this
• Some lessons require a more robust capability than ModSecurity's regular expressions, 'and/or' logical mechanisms, and goto statements (skip and skipAfter); again, Lua scripts are used to achieve this.
• One lesson uses the 'append' action to append JavaScript to the end of a response body, which alters the content and behavior of the HTML page

The rulesets can be used all together or, for a specific WebGoat sublesson, the initialization file (rulefile_00_initialize.conf) plus that sublesson's ruleset can be used.

The best way to open the discussion about the overall strategy used is to show a chunk of the initialization file:

<LocationMatch "^/WebGoat/attack$">
  # Group 1: the following block pertain to pages that don't have 
  #   Screen or menu parameters
1.  SecRule &ARGS:Screen "!@eq 0" chain,skipAfter:200
2.  SecRule &ARGS:menu "!@eq 0" "t:none"

  # Group 2: set session collection if entering WebGoat; POST body parameter 
  #   is "start=Start WebGoat" (Start WebGoat submit button)
3.  SecRule &ARGS_POST:start "@eq 0" "nolog,skip:3"
4.  SecRule ARGS_POST:start "!@streq Start WebGoat" \ 
      "t:urlDecodeUni,t:htmlEntityDecode,skip:2"
5.  SecRule REQUEST_COOKIES:JSESSIONID "!^$" \ 
      "chain,log,auditlog,pass,msg:'Setting session collection'"
6.  SecAction setsid:%{REQUEST_COOKIES.JSESSIONID} 
7.  SecAction "log,setvar:session.lesson13=0,msg:'setting session.lesson13=0 \  
      initially after setsid from rulefile_00-0-initialize.conf'"

8.  SecAction "t:none,allow,id:'200'"

  # Group 3: here there should be a 'menu' parameter, so set a variable for          
  #   the menu number that's used if needed in Phase 4
9.  SecRule ARGS_GET:menu "^(.*)$" "pass,setvar:tx.menu=%{MATCHED_VAR}"

In Group 1, the two chained rules denote the major section of WebGoat after the login and start pages. The 'Screen' parameter is arbitrary, while the 'menu' parameter is the lesson key. Basically, if this condition is met, skip past Group 2.

In Group 2, the next 2 rules pertain to the start page, denoted by the 'start=Start WebGoat' POST body parameter. Nothing of importance occurs on this page so we skip to the 'allow' action on line 8.

Line 5 and 6 denote that a successful login has occurred, so the session ID is saved in a session collection for later use throughout the remaining lessons. Line 7 sets a session variable (a toggle switch) for use in 'Lesson 13: Insecure Configuration'.

At Line 9, we have to save the 'menu' value for use in Phase 4.

At this point, we know the 'menu' value: it exists throughout phase 2 as is, and we have saved the value in a variable for Phase 4.

For the rest of the ruleset files, we will filter on the 'menu' parameter and each lesson will have its own ruleset file.

For example, ruleset file 'rulefile_04_authentication-flaws.conf' has 2 sections:

'SecRule ARGS:menu "!@eq 500" "phase:2,t:none,skip:2"' says "if we aren't in Lesson 4 in Phase 2, then don't do any further processing here".

Similarly, 'SecRule TX:MENU "!@eq 500" "phase:4,t:none,pass,skip:1"' says "if we aren't in Lesson 4 in Phase 2, then don't do any further processing here".

Reviewer comments

For many lesson solutions, a project reviewer has made comments regarding the project's choice of a solution (or lack of one), and often also has given a "classic" ModSecurity solution to those lessons solved by using Lua script.

Reviewer comments are added at the end of each individual lesson solution write-up in its own section; to date, the following sublessons have been updated with reviewer comments:

• 1. General -> 1.2 HTTP Splitting
• 2. Access Control Flaws -> 2.4 Remote Admin Access
• 4. Authentication Flaws -> 4.2 Forgot Password
• 4. Authentication Flaws -> 4.4 Multi Level Login 1
• 4. Authentication Flaws -> 4.5 Multi Level Login 2
• 6. Code Quality -> 6.1 Discover Clues in the HTML
• Lesson 8.0: 8. Cross-Site Scripting (XSS)] Addressing XSS attacks
• 8. Cross-Site Scripting (XSS) -> 8.2 LAB: Cross Site Scripting
• 8. Cross-Site Scripting (XSS) -> 8.4 Reflected XSS Attacks
• 8. Cross-Site Scripting (XSS) -> 8.5 Cross Site Request Forgery (CSRF)
• 8. Cross-Site Scripting (XSS) -> 8.6 HTTPOnly Test
• 8. Cross-Site Scripting (XSS) -> 8.7 Cross Site Tracing (XST) Attacks
• 16. Session Management Flaws -> 16.1 Hijack a Session
• 16. Session Management Flaws -> 16.2 Spoof an Authentication Cookie
• 16. Session Management Flaws -> 16.3 Session Fixation

Using the Lua scripting language

In some situations, the ModSecurity Rule Language is not flexible enough to prevent complex exploits. Enter Lua script, a tried-and-true programming language used mainly in the gaming industry but also used in other areas such as Nmap and the MySQL Proxy project.

Some advantages are:

1. Enables ModSecurity to address business logic flaws
2. Enhances the capability of using ModSecurity as an egress filter
3. Lua persistence: Super-persistence beyond the existing Session scope and capability (the SecDataDir directory is used for storage)
4. By being able to use a programming language, it is easier to do whitelisting and implement a positive security model.
5. Flexibility for virtual patching (as in, mitigating vulnerabilities discovered in a penetration test and NOT as in porting Snort or scanner signatures).
6. Ease-of-use: The Lua scripts can be developed and tested offline as standalone programs before being used by ModSecurity.
7. A programming language that is built to be rugged; e.g. handling multi-megabyte memory buffers and files.

In Phase 1, Lua scripts are used in sublessons:

• 4.2: Forgot Password (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_04.2)
• 4.4: Multi Level Login 1, 4.5: Multi Level Login 2 (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_04.4_04.5)
• 8.2 LAB: Cross Site Scripting (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_08.2_08.4_08.5_08.7)
• 15.1 Exploit Hidden Fields, 15.2 Exploit Unchecked Email (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_15.1_15.2)

In Phase 2, Lua scripts are used in sublessons:

• 3. AJAX Security -> 3.4 DOM Injection

• 3. AJAX Security -> 3.5 XML Injection

• 3. AJAX Security -> 3.6 JSON Injection

• 7. Concurrency -> 7.1 Thread Safety Problem

Shows the process of developing a standalone Lua script on Windows, then integrating it with ModSecurity on Linux

• 7. Concurrency -> 7.2 Shopping Cart Concurrency Flaw

Shows how Lua can be used to handle application state; in this case, a purchase that utilizes a shopping cart.

• 8. Cross-Site Scripting (XSS) -> 8.3 Stored XSS Attacks

Shows rudimentary output sanitization using Lua.

• 9. Denial of Service -> 9.1 Denial of Service from Multiple Logins

• 12. Insecure Communication -> 12.1 Insecure Login

Shows: (1) using JavaScript injection that adds an MD5 library on the front end in order to hash a password before being sent; (2) rebuilding the Lua engine and including a 3rd party MD5 library; and (3) hashing the expected password within a Lua script and comparing it with the client side password.

The Implementation section of each sublesson explains in detail how Lua script is used.

Using Javascript 'prepend' and 'append'

The Javascript 'prepend' and 'append' ModSecurity actions and their applications are shown in sublessons:

• 3. AJAX Security -> 3.1 LAB: DOM-Based cross-site scripting
• 3. AJAX Security -> 3.7 Silent Transactions Attacks
• 12. Insecure Communication -> 12.1 Insecure Login

Structure of mitigating a lesson

The following outlines the overall structure of each lesson/sublesson (they are used interchangeably throughout this project) to be used in the next section that contains the details of each mitigating solution:

Lesson overview: The path and file name of the lesson plan as taken from WebGoat. See Appendix A for more information.

Lesson solution: The path and file name of the lesson solution as taken from WebGoat. See Appendix A for more information.

Strategy (including challenges): What approach was taken to solve the lesson and why; what were the challenges? (if any)

Implementation: Details how the sublesson was mitigated.

Note that the project's solution files that are formatted using a Linux editor with a 2-space tab. For viewing the files from Windows, it's preferable to use Wordpad instead of Notepad because Notepad might add CR/LF combinations. Still, using Notepad and the files in DOS format on Linux should pose no problems but using Wordpad is recommended.

Comment(s): Any comments on the solution.

The mitigating solutions

Following are links to the mitigating solution pages. Some may link to the same page if their sublesson is solved by the same rule or by a group of rules in the same *.conf file.

Two asterisks (**) next to a sublesson denotes that it has not been solved but there are suggestions to solve it. Four asterisks (****) next to a sublesson indicates that there is no solution and no suggestion for it.

1. Sublesson 1.1: HTTP Basics
2. Sublesson 1.2: HTTP Splitting
3. Sublesson 2.2: Bypass a Path Based Access Control Scheme
4. Sublesson 2.3: LAB: Role Based Access Control **
5. Sublesson 2.4: Remote Admin Access
6. Sublesson 3.1: LAB: DOM-Based cross-site scripting
7. Sublesson 3.2: LAB: Client Side Filtering ****
8. Sublesson 3.4: DOM Injection
9. Sublesson 3.5: XML Injection
10. Sublesson 3.6: JSON Injection
11. Sublesson 3.7: Silent Transactions Attacks
12. Sublesson 3.8: Dangerous Use of Eval
13. Sublesson 3.9: Insecure Client Storage **
14. Sublesson 4.2: Forgot Password
15. Sublesson 4.4: Multi Level Login 1
16. Sublesson 4.5: Multi Level Login 2
17. Sublesson 6.1: Discover Clues in the HTML
18. Sublesson 7.1: Thread Safety Problem
19. Sublesson 7.2: Shopping Cart Concurrency Flaw
20. Lesson 8: Cross-Site Scripting (XSS) (Addressing XSS attacks - Reviewer comments)
21. Sublesson 8.1: Phishing with XSS
22. Sublesson 8.2: LAB: Cross Site Scripting
23. Sublesson 8.3: Stored XSS Attacks
24. Sublesson 8.4: Reflected XSS Attacks
25. Sublesson 8.5: Cross Site Request Forgery (CSRF)
26. Sublesson 8.7: Cross Site Tracing (XST) Attacks
27. Sublesson 8.6: HTTPOnly Test **
28. Sublesson 9.1: Denial of Service from Multiple Logins
29. Sublesson 10.1: Fail Open Authentication Scheme
30. Sublesson 11.1: Command Injection
31. Sublesson 11.2: Blind SQL Injection
32. Sublesson 11.3: Numeric SQL Injection
33. Sublesson 11.4: Log Spoofing
34. Sublesson 11.5: XPATH Injection
35. Sublesson 11.6: String SQL Injection
36. Sublesson 11.7: LAB: SQL Injection
37. Sublesson 11.8: Database Backdoors
38. Sublesson 12.1: Insecure Login
39. Sublesson 13.1: Forced Browsing
40. Sublesson 15.1: Exploit Hidden Fields
41. Sublesson 15.2: Exploit Unchecked Email
42. Sublesson 15.3: Bypass Client Side JavaScript Validation
43. Sublesson 16.1: Hijack a Session **
44. Sublesson 16.2: Spoof an Authentication Cookie **
45. Sublesson 16.3: Session Fixation **
46. Sublesson 17.1: Create a SOAP Request
47. Sublesson 17.2: WSDL Scanning **
48. Sublesson 17.3: Web Service SAX Injection
49. Sublesson 17.4: Web Service SQL Injection